-- Logs begin at Mon 2022-07-18 01:36:07 UTC, end at Mon 2022-07-18 02:53:34 UTC. -- Jul 18 02:13:51.480534 dv-cinder-850015-3-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 18 02:13:51.489137 dv-cinder-850015-3-iscsi devstack@keystone.service[80385]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 18 02:13:51.491194 dv-cinder-850015-3-iscsi devstack@keystone.service[80385]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Jul 18 02:13:51 2022] *** Jul 18 02:13:51.491194 dv-cinder-850015-3-iscsi devstack@keystone.service[80385]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 18 02:13:51.491194 dv-cinder-850015-3-iscsi devstack@keystone.service[80385]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 18 02:13:51.491194 dv-cinder-850015-3-iscsi devstack@keystone.service[80385]: nodename: dv-cinder-850015-3-iscsi Jul 18 02:13:51.491194 dv-cinder-850015-3-iscsi devstack@keystone.service[80385]: machine: x86_64 Jul 18 02:13:51.491194 dv-cinder-850015-3-iscsi devstack@keystone.service[80385]: clock source: unix Jul 18 02:13:51.491194 dv-cinder-850015-3-iscsi devstack@keystone.service[80385]: pcre jit disabled Jul 18 02:13:51.491194 dv-cinder-850015-3-iscsi devstack@keystone.service[80385]: detected number of CPU cores: 4 Jul 18 02:13:51.491194 dv-cinder-850015-3-iscsi devstack@keystone.service[80385]: current working directory: / Jul 18 02:13:51.491194 dv-cinder-850015-3-iscsi devstack@keystone.service[80385]: detected binary path: /usr/bin/uwsgi-core Jul 18 02:13:51.491194 dv-cinder-850015-3-iscsi devstack@keystone.service[80385]: your processes number limit is 47680 Jul 18 02:13:51.491832 dv-cinder-850015-3-iscsi devstack@keystone.service[80385]: your memory page size is 4096 bytes Jul 18 02:13:51.491832 dv-cinder-850015-3-iscsi devstack@keystone.service[80385]: detected max file descriptor number: 2048 Jul 18 02:13:51.491832 dv-cinder-850015-3-iscsi devstack@keystone.service[80385]: lock engine: pthread robust mutexes Jul 18 02:13:51.491832 dv-cinder-850015-3-iscsi devstack@keystone.service[80385]: thunder lock: enabled Jul 18 02:13:51.491832 dv-cinder-850015-3-iscsi devstack@keystone.service[80385]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 18 02:13:51.491832 dv-cinder-850015-3-iscsi devstack@keystone.service[80385]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Jul 18 02:13:51.531419 dv-cinder-850015-3-iscsi devstack@keystone.service[80385]: Python main interpreter initialized at 0x55b0b7b889c0 Jul 18 02:13:51.531419 dv-cinder-850015-3-iscsi devstack@keystone.service[80385]: python threads support enabled Jul 18 02:13:51.531419 dv-cinder-850015-3-iscsi devstack@keystone.service[80385]: your server socket listen backlog is limited to 100 connections Jul 18 02:13:51.531419 dv-cinder-850015-3-iscsi devstack@keystone.service[80385]: your mercy for graceful operations on workers is 90 seconds Jul 18 02:13:51.531774 dv-cinder-850015-3-iscsi devstack@keystone.service[80385]: mapped 403077 bytes (393 KB) for 2 cores Jul 18 02:13:51.531878 dv-cinder-850015-3-iscsi devstack@keystone.service[80385]: *** Operational MODE: preforking *** Jul 18 02:13:51.532148 dv-cinder-850015-3-iscsi devstack@keystone.service[80385]: *** uWSGI is running in multiple interpreter mode *** Jul 18 02:13:51.532148 dv-cinder-850015-3-iscsi devstack@keystone.service[80385]: spawned uWSGI master process (pid: 80385) Jul 18 02:13:51.532351 dv-cinder-850015-3-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 18 02:13:51.534179 dv-cinder-850015-3-iscsi devstack@keystone.service[80385]: spawned uWSGI worker 1 (pid: 80398, cores: 1) Jul 18 02:13:51.534179 dv-cinder-850015-3-iscsi devstack@keystone.service[80385]: spawned uWSGI worker 2 (pid: 80399, cores: 1) Jul 18 02:13:51.534480 dv-cinder-850015-3-iscsi devstack@keystone.service[80385]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 18 02:13:56.954934 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 18 02:13:56.954934 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: from cryptography.utils import int_from_bytes Jul 18 02:13:56.955786 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 18 02:13:56.955786 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: from cryptography.utils import int_from_bytes Jul 18 02:13:56.956222 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 18 02:13:56.956222 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: from cryptography.utils import int_from_bytes Jul 18 02:13:56.956335 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 18 02:13:56.956335 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: from cryptography.utils import int_from_bytes Jul 18 02:13:57.003828 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 18 02:13:57.003992 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 18 02:13:57.004086 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 18 02:13:57.004174 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 18 02:13:57.004255 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] command line args: [] {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 18 02:13:57.004334 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] command line args: [] {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 18 02:13:57.004443 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 18 02:13:57.004536 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 18 02:13:57.004610 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ================================================================================ {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 18 02:13:57.004688 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ================================================================================ {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 18 02:13:57.004854 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] admin_token = **** {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 02:13:57.004942 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] admin_token = **** {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 02:13:57.005094 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] config_dir = [] {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 02:13:57.005185 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] config_dir = [] {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 02:13:57.005266 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 02:13:57.005339 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 02:13:57.005444 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] config_source = [] {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 02:13:57.005525 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] config_source = [] {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 02:13:57.005604 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] debug = True {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 02:13:57.006246 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] debug = True {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 02:13:57.006461 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 02:13:57.006717 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 02:13:57.006815 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 02:13:57.006994 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 02:13:57.007086 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 02:13:57.007187 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 02:13:57.007269 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 02:13:57.007429 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 02:13:57.007516 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 02:13:57.007682 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 02:13:57.007792 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] list_limit = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 02:13:57.007895 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] log_config_append = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 02:13:57.007997 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] list_limit = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 02:13:57.008099 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 02:13:57.008197 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] log_config_append = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 02:13:57.008303 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] log_dir = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 02:13:57.008406 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 02:13:57.008508 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] log_file = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 02:13:57.008605 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] log_dir = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 02:13:57.008711 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 02:13:57.008788 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] log_file = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 02:13:57.008878 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 02:13:57.008953 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 02:13:57.009071 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 02:13:57.009147 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 02:13:57.009254 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 02:13:57.009342 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 02:13:57.009428 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 02:13:57.009503 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 02:13:57.009623 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 02:13:57.009700 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 02:13:57.009824 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 02:13:57.009910 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 02:13:57.010054 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 02:13:57.010180 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 02:13:57.010308 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 02:13:57.010389 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 02:13:57.010541 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 02:13:57.010674 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 02:13:57.010758 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 02:13:57.010832 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 02:13:57.010991 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 02:13:57.011121 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 02:13:57.011223 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 02:13:57.011311 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 02:13:57.011462 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 02:13:57.011589 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 02:13:57.011702 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 02:13:57.011787 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 02:13:57.011975 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 02:13:57.012201 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.66/identity {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 02:13:57.012381 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] publish_errors = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 02:13:57.012465 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.66/identity {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 02:13:57.012609 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 02:13:57.012687 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] publish_errors = False {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 02:13:57.012823 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 02:13:57.012908 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 02:13:57.012990 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 02:13:57.013070 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 02:13:57.013225 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 02:13:57.013312 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 02:13:57.013453 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 02:13:57.013544 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 02:13:57.013685 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] standard_threads = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 02:13:57.013771 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 02:13:57.013912 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 02:13:57.013999 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] standard_threads = False {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 02:13:57.014166 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 02:13:57.014264 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 02:13:57.014341 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 02:13:57.014412 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 02:13:57.014489 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] use_journal = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 02:13:57.014569 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 02:13:57.014647 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] use_json = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 02:13:57.014731 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] use_journal = False {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 02:13:57.014806 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] use_stderr = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 02:13:57.014931 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] use_json = False {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 02:13:57.015055 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] use_stderr = False {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 02:13:57.015134 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] use_syslog = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 02:13:57.015230 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] use_syslog = False {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 02:13:57.015386 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 02:13:57.015469 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 02:13:57.015610 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.015690 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.015851 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.015930 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.016107 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.016201 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.016344 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.016467 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.016591 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.016671 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.016827 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.016908 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.017070 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.017150 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.017303 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.017383 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.017546 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.017631 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.017787 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.017865 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.018031 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.018136 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.018287 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.018375 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.018531 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.018610 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.018764 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.018848 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.019004 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.019084 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.019246 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.019325 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.019480 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.019570 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.019729 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.019813 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.019981 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.020060 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.020222 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.020313 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.020454 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.020538 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.020701 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.020780 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.020957 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.021112 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.021194 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.021274 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.022006 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.022175 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.022421 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.022514 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.022656 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.022734 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.022886 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.022965 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.023121 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.023208 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.023367 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.023453 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.023614 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.023695 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.023844 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.023930 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.024072 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.024156 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.024316 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.024395 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.024562 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.024653 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.024793 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.024872 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.025036 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.025110 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.025265 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.025342 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.025497 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.025565 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.025726 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.025824 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.025951 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.026029 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.026202 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.026282 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.026429 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.026509 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.026655 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.026742 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.026888 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.026967 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.027131 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.027211 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.027358 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.027448 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.027592 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.027682 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.027817 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.027891 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.028047 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.028125 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.028277 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.028357 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.028520 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.028601 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.028747 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.028825 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.028980 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.029060 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.029212 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.029291 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.029442 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.029526 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.029675 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.029756 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.029907 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.029985 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.030157 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.030252 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.030403 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.030489 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.030634 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.030713 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.030870 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.030959 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.031116 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.031194 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.031402 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.031492 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.031640 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.031720 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.031875 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.031957 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.032111 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.032191 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.032345 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.032425 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.032609 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.032691 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.032847 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.032928 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.033092 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.033172 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.033325 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.033402 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.033557 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.033644 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.033788 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.033868 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.034020 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.034143 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.034292 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.034373 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.034528 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.034619 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.034763 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.034856 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.035002 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.035082 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.035235 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.035323 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.035469 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.035558 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.035701 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.035812 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.035933 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.036059 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.036164 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.036317 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.036424 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.036601 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.036681 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.036849 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.037088 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.037187 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.037264 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.037373 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.037454 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.037605 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.037684 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.037839 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.037921 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.038073 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.038192 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.038320 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.038401 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.038549 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.038637 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.038779 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.038989 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.039070 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.039241 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.039330 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.039507 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.039587 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.039751 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.039829 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.040024 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.040112 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.040291 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.040381 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.040540 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.040628 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.040807 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.040896 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.041066 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.041147 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.041306 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] auth.external = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.041387 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.041555 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.041648 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.041847 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.041935 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] auth.external = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.042139 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.042255 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.042387 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] auth.password = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.042470 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.042650 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] auth.token = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.042739 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.042925 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.043013 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] auth.password = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.043190 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.043277 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] auth.token = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.043466 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.043557 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.043705 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.043787 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.043955 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.044039 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.044188 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.044278 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.044437 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.044519 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.044685 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.044774 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] credential.caching = True {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.044917 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.045027 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.045152 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] credential.caching = True {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.045312 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.045402 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.045584 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.045665 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.045849 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.045931 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.046103 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.046195 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.046359 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.046442 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.046594 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.046677 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.046856 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.046937 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.047108 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.047196 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.047345 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.047468 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.047599 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.047785 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.047866 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.048058 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.048142 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.048353 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.048434 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.048593 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.048674 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.048850 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.048933 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.049084 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] federation.caching = True {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.049165 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] federation.caching = True {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.049320 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.049401 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.049564 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.049647 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.049808 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.049925 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.050039 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.050206 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.050301 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.050483 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.050571 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.050746 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.050827 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.050998 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.051086 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.051253 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.051596 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.051753 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.051834 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.052000 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.052093 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.052234 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] identity.caching = True {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.052316 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.052481 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.052719 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] identity.caching = True {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.052804 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.052889 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.052967 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.053156 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.053238 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.053414 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.053502 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.053642 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.053723 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.053879 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.053960 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.054147 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.054245 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.054399 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.054482 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.054632 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.054740 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.054864 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.055022 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.055114 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.055285 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.055367 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.055549 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.055630 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.055790 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.055882 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.056045 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.056137 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.056290 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.056378 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.056567 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.056658 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.056806 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.056887 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.057044 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.057125 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.057275 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.057357 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.057512 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.057593 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.057749 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.057831 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.057984 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.058068 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.058282 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.058363 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.058515 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.058596 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.058749 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.058834 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.059046 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.059126 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.059295 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.059376 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.059545 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.059628 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.059781 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.059862 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.060032 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.060113 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.060277 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.060358 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.060508 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.060589 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.060747 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.060828 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.060978 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.061061 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.061207 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.061293 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.061443 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.061523 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.061670 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.061751 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.061906 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.061987 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.062165 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.062251 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.062403 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.062486 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.062633 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.062713 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.062864 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.062946 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.063091 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.063172 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.063341 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.063424 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.063568 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.063649 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.063797 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.063880 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.064036 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.064116 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.064273 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.064354 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.064503 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.064585 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.user = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.064736 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.064818 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.064970 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.065053 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.065200 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.user = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.065283 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.065436 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.065519 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.065681 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.065761 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.065910 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.065992 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.066167 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.066249 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.066401 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.066483 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.066637 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.066721 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.066867 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.066947 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.067095 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.067177 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.067321 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.067403 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.067552 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.067638 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.067787 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.067868 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.068013 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.068096 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.068795 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.068878 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.069041 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.069245 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.069332 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.069409 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.069505 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.069587 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.069742 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.069823 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.069967 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.070049 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.070225 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.070308 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.070465 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.070548 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.070701 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.070784 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.070935 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.071017 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.071167 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.071248 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.071410 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.071493 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.071642 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.071743 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.071876 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.071964 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.072161 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.072246 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.072321 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.072395 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.072553 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.072648 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.072790 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.072870 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.073023 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.073105 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.073252 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.073340 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.073488 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.073569 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.073725 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.073805 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] resource.caching = True {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.073955 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.074048 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.074209 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.074293 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.074446 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] resource.caching = True {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.074528 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.074693 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.074786 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.074935 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.075015 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.075168 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.075247 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.075411 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.075499 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.075646 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.075727 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.075879 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.075959 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.076116 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] role.caching = True {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.076205 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.076352 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.076437 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] role.driver = sql {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.076580 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.076661 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.076811 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] role.caching = True {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.076900 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.077054 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] role.driver = sql {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.077136 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.077283 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.077364 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.077532 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.077620 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.077775 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.077856 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.078017 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.078151 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.078281 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.078375 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.078525 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.078609 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.078757 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.078839 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.079000 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.079089 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.079242 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.079322 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.079470 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.079550 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.079712 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.079802 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.079953 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.080107 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.080188 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.080350 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.080439 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.080598 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.080812 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.080894 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.081050 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.081132 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.081299 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.081383 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.081538 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.081618 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.081780 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.081861 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.082028 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.082130 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.082286 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.082369 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.082519 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.082598 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.082752 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.082836 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.082982 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.083063 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.083220 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.083300 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.083451 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.083533 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.083682 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.083789 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.084625 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.084748 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.084860 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.084990 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.085075 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.085150 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.085277 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] token.caching = True {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.085359 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.085515 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.085598 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.085747 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.085828 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.085981 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] token.caching = True {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.086063 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.086246 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.086333 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.086491 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.086573 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.086725 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.086805 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.086970 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.087054 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.087211 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.087291 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.087445 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.087527 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.087679 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.087767 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.087917 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.087998 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.088151 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.088233 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.088384 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.088466 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.088620 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.088703 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.088852 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.088934 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.089092 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.089174 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.089340 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.089420 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.089569 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.089651 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] database.connection = **** {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.089811 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.089906 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.090122 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.090207 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.090366 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.090448 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] database.connection = **** {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.090613 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.090706 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.090841 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.090924 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.091075 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.091156 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.091309 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.091389 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.091544 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.091623 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.091760 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.091842 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.091995 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.092076 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.092229 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.092312 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.092461 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.092541 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.092694 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.092774 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.092929 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.093010 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.093159 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.093240 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.093402 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.093482 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.093636 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.093717 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.093867 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.093947 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.094022 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 18 02:13:57.094115 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.094350 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.094585 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 02:13:57.094753 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 18 02:13:57.124560 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 02:13:57.124720 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 02:13:57.124795 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 02:13:57.124890 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 02:13:57.125000 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 02:13:57.125087 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 02:13:57.134005 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 02:13:57.135020 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 02:13:57.162755 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 02:13:57.162990 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 02:13:57.163454 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 02:13:57.163685 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 02:13:57.305899 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 02:13:57.306716 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 02:13:57.355228 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 02:13:57.355461 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 02:13:57.356053 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 02:13:57.356295 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 02:13:57.420450 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 02:13:57.420665 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 02:13:57.420778 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 02:13:57.420891 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 02:13:57.420978 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 02:13:57.421111 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 02:13:57.421208 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 02:13:57.421327 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 02:13:57.421411 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 02:13:57.421543 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 02:13:57.421627 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 02:13:57.421759 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 02:13:57.448035 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 02:13:57.448266 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 02:13:57.448355 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 02:13:57.448490 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 02:13:57.448593 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 02:13:57.448707 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 02:13:57.448792 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 02:13:57.448920 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 02:13:57.449005 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 02:13:57.449129 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 02:13:57.449223 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 02:13:57.449361 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 02:13:57.449455 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 02:13:57.449615 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 02:13:57.460540 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 02:13:57.460986 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 02:13:57.473146 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 02:13:57.473381 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 02:13:57.473469 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 02:13:57.473606 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 02:13:57.473706 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 02:13:57.473825 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 02:13:57.473917 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 02:13:57.474042 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 02:13:57.474146 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 02:13:57.474285 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 02:13:57.474378 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 02:13:57.474555 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 02:13:57.474644 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 02:13:57.474867 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 02:13:57.492485 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.492694 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 18 02:13:57.492781 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.492973 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 18 02:13:57.493056 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.493213 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 18 02:13:57.493305 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.493479 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.493562 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 18 02:13:57.493747 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.493886 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.494021 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.494198 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.494331 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.494465 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.494597 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.505213 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.505490 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.505620 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.506440 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.506581 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.506713 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.506915 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.507069 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.514982 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 02:13:57.515119 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80398) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 02:13:57.518264 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 02:13:57.518405 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80398) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 02:13:57.518546 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.518680 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.518808 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.518948 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.519178 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.519327 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.519463 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.519595 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.519727 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.519858 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.520074 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.520216 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.520349 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.520477 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.520610 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.520741 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.520890 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.521023 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.521154 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.521321 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.547653 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.547840 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.549263 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 02:13:57.549467 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80398) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 02:13:57.549599 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.549736 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.556032 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 02:13:57.556191 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80398) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 02:13:57.556323 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.556456 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.556588 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.556720 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.563750 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 02:13:57.563847 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80398) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 02:13:57.564032 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.564116 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.568713 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80398) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 02:13:57.568892 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 02:13:57.569058 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.569177 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.569341 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.569455 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.569625 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.569733 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.569899 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.570004 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.570193 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.570300 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.570464 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.570570 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.570731 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.570835 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.589417 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.589691 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.589784 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.590042 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.594230 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80398) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 02:13:57.594614 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 02:13:57.597944 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.598260 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.598378 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.598571 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.603804 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80398) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 02:13:57.604229 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 02:13:57.607473 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80398) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 02:13:57.607886 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 02:13:57.611236 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.611532 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.611647 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.611822 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.611919 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.612106 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.612200 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.612405 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.612507 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.612695 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.612798 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.613004 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.632285 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80398) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 02:13:57.632389 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 02:13:57.632661 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.632746 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.632930 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.633015 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.633201 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.633285 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.641088 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.641203 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.642828 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.642925 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.643103 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.643188 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.646163 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80398) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 02:13:57.646263 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 02:13:57.646543 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.646746 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.646828 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.646905 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.647093 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.647178 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.647382 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.647467 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.660776 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 02:13:57.660913 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80398) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 02:13:57.661052 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.661193 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.661326 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.661471 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.661603 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.661748 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.662436 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.662616 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.662782 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.662848 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.662988 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.663150 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.682932 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 02:13:57.683098 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80398) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 02:13:57.686181 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 02:13:57.686374 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80398) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 02:13:57.689452 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.689661 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.691069 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.691303 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.692703 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 02:13:57.692969 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80398) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 02:13:57.693113 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.693801 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.693965 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.694235 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.701152 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.701612 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.702772 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 02:13:57.703255 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80398) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 02:13:57.705978 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.706296 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.706517 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.706651 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.706813 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.706945 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.707089 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.707357 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.716554 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 02:13:57.716838 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.717121 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.717262 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80398) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 02:13:57.717482 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.717762 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.725003 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 02:13:57.725277 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.725556 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.725686 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80398) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 02:13:57.725815 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.725974 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.726131 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.726262 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.726402 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.726533 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.726663 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.726793 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.726921 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.727060 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.727190 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.727320 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.727463 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.727595 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.727733 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.727863 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.727992 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.728123 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.728253 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.728381 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.728519 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.728649 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.728788 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.728982 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 02:13:57.766617 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55b0b7b889c0 pid: 80399 (default app) Jul 18 02:13:57.767161 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55b0b7b889c0 pid: 80398 (default app) Jul 18 02:13:57.769933 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70723ba-bfc2-4810-9525-284cd0ec111b None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:13:57.770246 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70723ba-bfc2-4810-9525-284cd0ec111b None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:13:57.770523 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70723ba-bfc2-4810-9525-284cd0ec111b None None] PATH_INFO: `/v3` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:13:57.775356 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1/1] 10.222.0.66 () {50 vars in 791 bytes} [Mon Jul 18 02:13:57 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:14:06.317497 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40076cf6-a6a0-4d55-856b-fbd1a85a4379 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:06.318200 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40076cf6-a6a0-4d55-856b-fbd1a85a4379 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:06.318326 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40076cf6-a6a0-4d55-856b-fbd1a85a4379 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:06.322656 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1/2] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:14:06 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:14:06.327919 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1856f690-6088-41e1-ac1c-5e73e8fe485c None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:06.328329 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1856f690-6088-41e1-ac1c-5e73e8fe485c None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:06.328472 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1856f690-6088-41e1-ac1c-5e73e8fe485c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:06.397684 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: INFO dbcounter [None req-1856f690-6088-41e1-ac1c-5e73e8fe485c None None] Registered counter for database keystone Jul 18 02:14:06.419561 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG oslo_db.sqlalchemy.engines [None req-1856f690-6088-41e1-ac1c-5e73e8fe485c None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80399) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 18 02:14:06.425909 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writer thread running {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 18 02:14:06.604698 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG passlib.handlers.bcrypt [None req-1856f690-6088-41e1-ac1c-5e73e8fe485c None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80399) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 18 02:14:06.605047 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG passlib.handlers.bcrypt [None req-1856f690-6088-41e1-ac1c-5e73e8fe485c None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80399) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 18 02:14:06.659116 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-1856f690-6088-41e1-ac1c-5e73e8fe485c None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:14:06.660716 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-1856f690-6088-41e1-ac1c-5e73e8fe485c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:14:06.783216 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 2/3] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:14:06 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 456 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 18 02:14:06.931408 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40076cf6-a6a0-4d55-856b-fbd1a85a4379 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:07.000019 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: INFO dbcounter [None req-40076cf6-a6a0-4d55-856b-fbd1a85a4379 None None] Registered counter for database keystone Jul 18 02:14:07.019285 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG oslo_db.sqlalchemy.engines [None req-40076cf6-a6a0-4d55-856b-fbd1a85a4379 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80398) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 18 02:14:07.025983 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writer thread running {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 18 02:14:07.148903 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40076cf6-a6a0-4d55-856b-fbd1a85a4379 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:07.153252 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf11ab3-847f-414f-919c-a729b237e906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:07.153915 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf11ab3-847f-414f-919c-a729b237e906 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:07.154260 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf11ab3-847f-414f-919c-a729b237e906 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:07.154529 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf11ab3-847f-414f-919c-a729b237e906 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:07.163652 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf11ab3-847f-414f-919c-a729b237e906 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:07.216068 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: WARNING keystone.common.rbac_enforcer.enforcer [None req-fdf11ab3-847f-414f-919c-a729b237e906 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 18 02:14:07.216355 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf11ab3-847f-414f-919c-a729b237e906 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:07.226841 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 2/4] 10.222.0.66 () {58 vars in 1324 bytes} [Mon Jul 18 02:14:06 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 18 02:14:07.232843 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1856f690-6088-41e1-ac1c-5e73e8fe485c None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:07.247591 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1856f690-6088-41e1-ac1c-5e73e8fe485c None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:07.252077 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d1f778-4a06-4f7c-90e0-deb0476cde29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:07.253182 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d1f778-4a06-4f7c-90e0-deb0476cde29 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:07.253471 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d1f778-4a06-4f7c-90e0-deb0476cde29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:07.253749 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d1f778-4a06-4f7c-90e0-deb0476cde29 admin admin] PATH_INFO: `/v3/projects/a05e40a15656441b94f5763e4d68d305` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:07.255290 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d1f778-4a06-4f7c-90e0-deb0476cde29 admin admin] RBAC: Authorizing `identity:get_project(project_id=a05e40a15656441b94f5763e4d68d305)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:07.308854 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: WARNING keystone.common.rbac_enforcer.enforcer [None req-45d1f778-4a06-4f7c-90e0-deb0476cde29 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 18 02:14:07.309201 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d1f778-4a06-4f7c-90e0-deb0476cde29 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:07.313968 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 3/5] 10.222.0.66 () {56 vars in 1207 bytes} [Mon Jul 18 02:14:07 2022] GET /identity/v3/projects/a05e40a15656441b94f5763e4d68d305 => generated 344 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:14:09.298796 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1efff0a-81fd-4eee-8720-a6e620beb705 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:09.299355 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1efff0a-81fd-4eee-8720-a6e620beb705 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:09.299355 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1efff0a-81fd-4eee-8720-a6e620beb705 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:09.304848 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 3/6] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:14:09 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 02:14:09.310485 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bb5bca-d888-44ad-b27b-3c0747df0cbe None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:09.310935 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bb5bca-d888-44ad-b27b-3c0747df0cbe None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:09.311117 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bb5bca-d888-44ad-b27b-3c0747df0cbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:09.330059 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-e6bb5bca-d888-44ad-b27b-3c0747df0cbe None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:14:09.331608 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-e6bb5bca-d888-44ad-b27b-3c0747df0cbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:14:09.339896 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 4/7] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:14:09 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 18 02:14:09.362042 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1efff0a-81fd-4eee-8720-a6e620beb705 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:09.374268 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1efff0a-81fd-4eee-8720-a6e620beb705 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:09.378471 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0528c10b-0b35-40f7-b020-b18df512aa22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:09.379108 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0528c10b-0b35-40f7-b020-b18df512aa22 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:09.379397 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0528c10b-0b35-40f7-b020-b18df512aa22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:09.379679 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0528c10b-0b35-40f7-b020-b18df512aa22 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:09.387451 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0528c10b-0b35-40f7-b020-b18df512aa22 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:09.389242 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0528c10b-0b35-40f7-b020-b18df512aa22 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:09.398508 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 4/8] 10.222.0.66 () {58 vars in 1324 bytes} [Mon Jul 18 02:14:09 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 02:14:09.403563 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bb5bca-d888-44ad-b27b-3c0747df0cbe None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:09.412620 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bb5bca-d888-44ad-b27b-3c0747df0cbe None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:09.416902 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7b4ca7-be81-4bd7-a1a4-281593af8f5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:09.417746 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7b4ca7-be81-4bd7-a1a4-281593af8f5a admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:09.418026 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7b4ca7-be81-4bd7-a1a4-281593af8f5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:09.418330 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7b4ca7-be81-4bd7-a1a4-281593af8f5a admin admin] PATH_INFO: `/v3/users/62dd67591e0147fe97b248079359a4de` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:09.419695 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7b4ca7-be81-4bd7-a1a4-281593af8f5a admin admin] RBAC: Authorizing `identity:get_user(user_id=62dd67591e0147fe97b248079359a4de)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:09.421507 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7b4ca7-be81-4bd7-a1a4-281593af8f5a admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:09.423125 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 5/9] 10.222.0.66 () {56 vars in 1198 bytes} [Mon Jul 18 02:14:09 2022] GET /identity/v3/users/62dd67591e0147fe97b248079359a4de => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:14:11.463815 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d0f3c3-394a-4b02-ab2f-298f0d2cf2a8 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:11.464376 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d0f3c3-394a-4b02-ab2f-298f0d2cf2a8 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:11.464376 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d0f3c3-394a-4b02-ab2f-298f0d2cf2a8 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:11.465709 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 5/10] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:14:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:14:11.470947 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9a68a4-53e1-44ce-9917-aa3745fb48a7 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:11.471387 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9a68a4-53e1-44ce-9917-aa3745fb48a7 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:11.471491 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9a68a4-53e1-44ce-9917-aa3745fb48a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:11.488082 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-5d9a68a4-53e1-44ce-9917-aa3745fb48a7 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:14:11.490142 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-5d9a68a4-53e1-44ce-9917-aa3745fb48a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:14:11.522289 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea0b758-b824-42dd-84e8-fdaa14f855f3 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:11.522582 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea0b758-b824-42dd-84e8-fdaa14f855f3 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:11.522861 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea0b758-b824-42dd-84e8-fdaa14f855f3 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:11.524225 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 6/11] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:14:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 02:14:11.529231 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73abecd-6105-44d7-a9c4-eed74ee0c414 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:11.529518 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73abecd-6105-44d7-a9c4-eed74ee0c414 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:11.529796 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73abecd-6105-44d7-a9c4-eed74ee0c414 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:11.570760 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG passlib.handlers.bcrypt [None req-a73abecd-6105-44d7-a9c4-eed74ee0c414 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80398) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 18 02:14:11.571177 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG passlib.handlers.bcrypt [None req-a73abecd-6105-44d7-a9c4-eed74ee0c414 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80398) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 18 02:14:11.572307 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 6/12] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:14:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 02:14:11.594935 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9a68a4-53e1-44ce-9917-aa3745fb48a7 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:11.616794 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-a73abecd-6105-44d7-a9c4-eed74ee0c414 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:14:11.618425 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-a73abecd-6105-44d7-a9c4-eed74ee0c414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:14:11.621865 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:11.701450 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9a68a4-53e1-44ce-9917-aa3745fb48a7 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:11.719277 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 7/13] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:14:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 02:14:11.735197 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec6f34d-6e1b-444f-a489-6e052c3a4593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:11.735999 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec6f34d-6e1b-444f-a489-6e052c3a4593 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:11.736286 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec6f34d-6e1b-444f-a489-6e052c3a4593 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:11.736567 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec6f34d-6e1b-444f-a489-6e052c3a4593 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:11.741186 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73abecd-6105-44d7-a9c4-eed74ee0c414 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:11.750162 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec6f34d-6e1b-444f-a489-6e052c3a4593 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 02:14:11.762810 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-6ec6f34d-6e1b-444f-a489-6e052c3a4593 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 02:14:11.762810 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:14:11.762810 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:14:11.762810 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:14:11.762810 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:14:11.762810 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:14:11.762810 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:14:11.762810 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:14:11.762810 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:14:11.762810 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:14:11.762810 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:14:11.762810 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:14:11.762810 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 02:14:11.762810 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 02:14:11.762810 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 02:14:11.762810 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 02:14:11.763905 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 02:14:11.763905 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 02:14:11.763905 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 02:14:11.763905 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 02:14:11.763905 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:14:11.763905 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:14:11.763905 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:14:11.763905 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:14:11.763905 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:14:11.763905 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:14:11.763905 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:14:11.763905 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:14:11.763905 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:14:11.763905 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:14:11.763905 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:14:11.763905 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:14:11.764902 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:14:11.764902 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:14:11.764902 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:14:11.764902 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:14:11.764902 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 02:14:11.764902 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 02:14:11.764902 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 02:14:11.764902 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 02:14:11.764902 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 02:14:11.764902 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 02:14:11.764902 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 02:14:11.764902 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:14:11.764902 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 7/14] 10.222.0.66 () {56 vars in 1096 bytes} [Mon Jul 18 02:14:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 02:14:11.769709 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec6f34d-6e1b-444f-a489-6e052c3a4593 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:11.842304 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73abecd-6105-44d7-a9c4-eed74ee0c414 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:11.844728 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:11.851401 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:11.869111 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec6f34d-6e1b-444f-a489-6e052c3a4593 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:11.879320 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ce3795-2ed4-41ef-941b-6e76801af30d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:11.880959 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ce3795-2ed4-41ef-941b-6e76801af30d None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:11.881246 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ce3795-2ed4-41ef-941b-6e76801af30d None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:11.881516 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ce3795-2ed4-41ef-941b-6e76801af30d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:11.904654 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b350a7-f5ab-423b-b43c-e4cc2af3946d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:11.905319 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b350a7-f5ab-423b-b43c-e4cc2af3946d None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:11.905605 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b350a7-f5ab-423b-b43c-e4cc2af3946d None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:11.905886 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b350a7-f5ab-423b-b43c-e4cc2af3946d None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:11.906665 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b350a7-f5ab-423b-b43c-e4cc2af3946d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:14:11.906975 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b350a7-f5ab-423b-b43c-e4cc2af3946d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:11.908438 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b350a7-f5ab-423b-b43c-e4cc2af3946d None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:11.920601 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 8/15] 10.222.0.66 () {56 vars in 1099 bytes} [Mon Jul 18 02:14:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 02:14:11.926509 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b350a7-f5ab-423b-b43c-e4cc2af3946d None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:11.948922 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ce3795-2ed4-41ef-941b-6e76801af30d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:11.950496 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ce3795-2ed4-41ef-941b-6e76801af30d None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:12.028727 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b350a7-f5ab-423b-b43c-e4cc2af3946d None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:12.049465 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 8/16] 10.222.0.66 () {58 vars in 1282 bytes} [Mon Jul 18 02:14:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 18 02:14:12.055390 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ce3795-2ed4-41ef-941b-6e76801af30d None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:12.058171 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:12.065064 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebfed13-6940-4610-9144-66b6ea5e8091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:12.065940 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebfed13-6940-4610-9144-66b6ea5e8091 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:12.066247 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebfed13-6940-4610-9144-66b6ea5e8091 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:12.067576 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebfed13-6940-4610-9144-66b6ea5e8091 None admin] PATH_INFO: `/v3/users/62dd67591e0147fe97b248079359a4de` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:12.068511 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebfed13-6940-4610-9144-66b6ea5e8091 None admin] RBAC: Authorizing `identity:get_user(user_id=62dd67591e0147fe97b248079359a4de)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:12.069983 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebfed13-6940-4610-9144-66b6ea5e8091 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:12.071773 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 9/17] 10.222.0.66 () {56 vars in 1177 bytes} [Mon Jul 18 02:14:11 2022] GET /identity/v3/users/62dd67591e0147fe97b248079359a4de => generated 247 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 02:14:12.077003 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebfed13-6940-4610-9144-66b6ea5e8091 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:12.092035 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:12.155082 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ce3795-2ed4-41ef-941b-6e76801af30d None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:12.178161 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebfed13-6940-4610-9144-66b6ea5e8091 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:12.188683 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f650438b-6003-439f-ab19-f7dcdf0e6ecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:12.189552 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f650438b-6003-439f-ab19-f7dcdf0e6ecc None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:12.189929 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f650438b-6003-439f-ab19-f7dcdf0e6ecc None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:12.190310 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f650438b-6003-439f-ab19-f7dcdf0e6ecc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:12.210435 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f650438b-6003-439f-ab19-f7dcdf0e6ecc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:12.211922 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f650438b-6003-439f-ab19-f7dcdf0e6ecc None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:12.212547 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f13d42f-ac6b-4968-83bf-805c5c3bcece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:12.213350 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f13d42f-ac6b-4968-83bf-805c5c3bcece None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:12.213637 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f13d42f-ac6b-4968-83bf-805c5c3bcece None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:12.213912 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f13d42f-ac6b-4968-83bf-805c5c3bcece None admin] PATH_INFO: `/v3/domains/default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:12.215149 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f13d42f-ac6b-4968-83bf-805c5c3bcece None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:12.216623 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f13d42f-ac6b-4968-83bf-805c5c3bcece None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:12.218364 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 10/18] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 18 02:14:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 02:14:12.224030 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f13d42f-ac6b-4968-83bf-805c5c3bcece None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:12.232925 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-f650438b-6003-439f-ab19-f7dcdf0e6ecc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:14:12.232925 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:14:12.232925 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:14:12.232925 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:14:12.232925 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:14:12.232925 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:14:12.232925 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:14:12.232925 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:14:12.232925 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:14:12.232925 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:14:12.232925 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:14:12.232925 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:14:12.232925 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 02:14:12.232925 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 02:14:12.232925 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 02:14:12.233921 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 02:14:12.233921 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:14:12.233921 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:14:12.233921 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:14:12.233921 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:14:12.233921 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:14:12.233921 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:14:12.233921 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:14:12.233921 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:14:12.233921 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:14:12.233921 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:14:12.233921 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:14:12.233921 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:14:12.233921 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:14:12.233921 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:14:12.233921 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:14:12.233921 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:14:12.234797 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 02:14:12.234797 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 02:14:12.234797 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:14:12.234797 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:14:12.234797 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 9/19] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 18 02:14:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 02:14:12.240607 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f650438b-6003-439f-ab19-f7dcdf0e6ecc None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:12.295542 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:12.304998 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:12.324321 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f13d42f-ac6b-4968-83bf-805c5c3bcece None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:12.339124 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f650438b-6003-439f-ab19-f7dcdf0e6ecc None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:12.357372 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7023654-00b9-481e-9f82-6d0727386297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:12.358017 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7023654-00b9-481e-9f82-6d0727386297 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:12.358327 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7023654-00b9-481e-9f82-6d0727386297 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:12.358604 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7023654-00b9-481e-9f82-6d0727386297 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:12.359430 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7023654-00b9-481e-9f82-6d0727386297 None admin] RBAC: Adding query filter params (role.id=d4aa3324f0c84887b41d0dc4acf7f30e, scope.domain.id=default, user.id=62dd67591e0147fe97b248079359a4de) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:14:12.359781 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7023654-00b9-481e-9f82-6d0727386297 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:12.362233 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7023654-00b9-481e-9f82-6d0727386297 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:12.370098 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 11/20] 10.222.0.66 () {56 vars in 1322 bytes} [Mon Jul 18 02:14:12 2022] GET /identity/v3/role_assignments?user.id=62dd67591e0147fe97b248079359a4de&scope.domain.id=default&role.id=d4aa3324f0c84887b41d0dc4acf7f30e => generated 233 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 02:14:12.373522 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9005d4d-3db8-4c79-b273-3d003f7ac07f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:12.374231 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9005d4d-3db8-4c79-b273-3d003f7ac07f None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:12.374516 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9005d4d-3db8-4c79-b273-3d003f7ac07f None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:12.374790 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9005d4d-3db8-4c79-b273-3d003f7ac07f None admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:12.375532 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9005d4d-3db8-4c79-b273-3d003f7ac07f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:14:12.375838 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9005d4d-3db8-4c79-b273-3d003f7ac07f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:12.377276 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9005d4d-3db8-4c79-b273-3d003f7ac07f None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:12.398047 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 10/21] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 18 02:14:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 02:14:14.592521 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc26a595-ece4-46ed-92eb-3d54c221e916 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:14.593070 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc26a595-ece4-46ed-92eb-3d54c221e916 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:14.593070 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc26a595-ece4-46ed-92eb-3d54c221e916 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:14.594666 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 12/22] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:14:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:14:14.603895 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8e4100-81b4-46d3-a986-5c52628097b5 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:14.604830 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8e4100-81b4-46d3-a986-5c52628097b5 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:14.605329 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8e4100-81b4-46d3-a986-5c52628097b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:14.639106 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-6b8e4100-81b4-46d3-a986-5c52628097b5 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:14:14.640316 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-6b8e4100-81b4-46d3-a986-5c52628097b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:14:14.699648 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12cf9a5-d231-4173-be45-e4436740915b None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:14.699980 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12cf9a5-d231-4173-be45-e4436740915b None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:14.700227 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12cf9a5-d231-4173-be45-e4436740915b None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:14.701617 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 13/23] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:14:14 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 02:14:14.706798 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615a3baa-7fb2-4af5-905c-90ecd5a9a9d5 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:14.707092 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615a3baa-7fb2-4af5-905c-90ecd5a9a9d5 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:14.707396 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615a3baa-7fb2-4af5-905c-90ecd5a9a9d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:14.741420 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 11/24] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:14:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 02:14:14.755676 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-615a3baa-7fb2-4af5-905c-90ecd5a9a9d5 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:14:14.756910 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-615a3baa-7fb2-4af5-905c-90ecd5a9a9d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:14:14.786183 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8e4100-81b4-46d3-a986-5c52628097b5 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:14.896046 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:14.908010 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 14/25] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:14:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 02:14:14.948973 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615a3baa-7fb2-4af5-905c-90ecd5a9a9d5 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:15.026525 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8e4100-81b4-46d3-a986-5c52628097b5 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:15.077086 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:15.111368 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615a3baa-7fb2-4af5-905c-90ecd5a9a9d5 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:15.149854 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81aa542b-5610-4768-b4f8-ca5d35684178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:15.150673 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aa542b-5610-4768-b4f8-ca5d35684178 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:15.150960 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aa542b-5610-4768-b4f8-ca5d35684178 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:15.151245 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aa542b-5610-4768-b4f8-ca5d35684178 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:15.169734 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaacf62-bef6-4331-9e2f-665ba27041a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:15.170964 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaacf62-bef6-4331-9e2f-665ba27041a1 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:15.171471 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaacf62-bef6-4331-9e2f-665ba27041a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:15.171945 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaacf62-bef6-4331-9e2f-665ba27041a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:15.185354 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81aa542b-5610-4768-b4f8-ca5d35684178 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 02:14:15.195526 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-81aa542b-5610-4768-b4f8-ca5d35684178 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 02:14:15.195526 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:14:15.195526 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:14:15.195526 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:14:15.195526 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:14:15.195526 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:14:15.195526 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:14:15.195526 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:14:15.195526 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:14:15.195526 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:14:15.195526 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:14:15.195526 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:14:15.195526 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 02:14:15.195526 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 02:14:15.195526 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 02:14:15.195526 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 02:14:15.196362 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 02:14:15.196362 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 02:14:15.196362 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 02:14:15.196362 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 02:14:15.196362 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:14:15.196362 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:14:15.196362 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:14:15.196362 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:14:15.196362 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:14:15.196362 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:14:15.196362 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:14:15.196362 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:14:15.196362 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:14:15.196362 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:14:15.196362 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:14:15.196362 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:14:15.197166 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:14:15.197166 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:14:15.197166 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:14:15.197166 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:14:15.197166 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 02:14:15.197166 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 02:14:15.197166 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 02:14:15.197166 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 02:14:15.197166 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 02:14:15.197166 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 02:14:15.197166 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 02:14:15.197166 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:14:15.202262 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efaacf62-bef6-4331-9e2f-665ba27041a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:15.204009 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efaacf62-bef6-4331-9e2f-665ba27041a1 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:15.213613 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 12/26] 10.222.0.66 () {56 vars in 1096 bytes} [Mon Jul 18 02:14:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 429 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 02:14:15.219504 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81aa542b-5610-4768-b4f8-ca5d35684178 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:15.234873 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-efaacf62-bef6-4331-9e2f-665ba27041a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:14:15.234873 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:14:15.234873 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:14:15.234873 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:14:15.234873 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:14:15.234873 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:14:15.234873 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:14:15.234873 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:14:15.234873 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:14:15.234873 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:14:15.234873 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:14:15.234873 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:14:15.234873 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 02:14:15.234873 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 02:14:15.234873 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 02:14:15.235771 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 02:14:15.235771 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:14:15.235771 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:14:15.235771 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:14:15.235771 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:14:15.235771 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:14:15.235771 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:14:15.235771 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:14:15.235771 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:14:15.235771 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:14:15.235771 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:14:15.235771 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:14:15.235771 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:14:15.235771 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:14:15.235771 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:14:15.235771 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:14:15.235771 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:14:15.237032 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 02:14:15.237032 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 02:14:15.237032 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:14:15.237032 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:14:15.237032 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 15/27] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 18 02:14:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 02:14:15.243535 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaacf62-bef6-4331-9e2f-665ba27041a1 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:15.343999 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81aa542b-5610-4768-b4f8-ca5d35684178 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:15.358463 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:15.421778 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647d3a2a-af50-4821-924c-228abb11f7d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:15.422876 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647d3a2a-af50-4821-924c-228abb11f7d7 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:15.423362 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647d3a2a-af50-4821-924c-228abb11f7d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:15.423837 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647d3a2a-af50-4821-924c-228abb11f7d7 None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:15.427236 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647d3a2a-af50-4821-924c-228abb11f7d7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:14:15.432147 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647d3a2a-af50-4821-924c-228abb11f7d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:15.433915 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647d3a2a-af50-4821-924c-228abb11f7d7 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:15.457289 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaacf62-bef6-4331-9e2f-665ba27041a1 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:15.460299 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 13/28] 10.222.0.66 () {56 vars in 1099 bytes} [Mon Jul 18 02:14:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 02:14:15.466032 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647d3a2a-af50-4821-924c-228abb11f7d7 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:15.511537 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:15.520713 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b812a15-f019-4541-a54f-7da120a7f837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:15.523975 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b812a15-f019-4541-a54f-7da120a7f837 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:15.523975 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b812a15-f019-4541-a54f-7da120a7f837 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:15.523975 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b812a15-f019-4541-a54f-7da120a7f837 None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:15.523975 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b812a15-f019-4541-a54f-7da120a7f837 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:14:15.523975 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b812a15-f019-4541-a54f-7da120a7f837 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:15.524921 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b812a15-f019-4541-a54f-7da120a7f837 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:15.544735 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 16/29] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 18 02:14:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:14:15.546666 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672db21f-072c-4134-a6e8-cf825011a0e4 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:15.547045 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672db21f-072c-4134-a6e8-cf825011a0e4 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:15.547386 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672db21f-072c-4134-a6e8-cf825011a0e4 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:15.548718 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 17/30] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:14:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:14:15.554856 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17133331-389b-44ce-a11b-5d6977f32f58 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:15.555246 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17133331-389b-44ce-a11b-5d6977f32f58 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:15.555552 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17133331-389b-44ce-a11b-5d6977f32f58 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:15.557228 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 18/31] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:14:15 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:14:15.557795 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17133331-389b-44ce-a11b-5d6977f32f58 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:15.595715 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647d3a2a-af50-4821-924c-228abb11f7d7 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:15.637918 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eea94d1-ebb5-4c12-be58-76bd63d22a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:15.640158 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eea94d1-ebb5-4c12-be58-76bd63d22a0e None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:15.640638 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eea94d1-ebb5-4c12-be58-76bd63d22a0e None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:15.641236 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eea94d1-ebb5-4c12-be58-76bd63d22a0e None admin] PATH_INFO: `/v3/users/62dd67591e0147fe97b248079359a4de` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:15.643550 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eea94d1-ebb5-4c12-be58-76bd63d22a0e None admin] RBAC: Authorizing `identity:get_user(user_id=62dd67591e0147fe97b248079359a4de)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:15.646272 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eea94d1-ebb5-4c12-be58-76bd63d22a0e None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:15.648822 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 14/32] 10.222.0.66 () {56 vars in 1177 bytes} [Mon Jul 18 02:14:15 2022] GET /identity/v3/users/62dd67591e0147fe97b248079359a4de => generated 247 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:14:15.650532 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb05b045-2f35-49cd-b484-89f6e82ee576 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:15.651485 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb05b045-2f35-49cd-b484-89f6e82ee576 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:15.651997 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb05b045-2f35-49cd-b484-89f6e82ee576 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:15.673000 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-eb05b045-2f35-49cd-b484-89f6e82ee576 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:14:15.674539 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-eb05b045-2f35-49cd-b484-89f6e82ee576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:14:15.692856 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17133331-389b-44ce-a11b-5d6977f32f58 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:15.713504 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:15.739531 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a3c48b-66da-40a8-b96b-44ae12ab1155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:15.740251 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a3c48b-66da-40a8-b96b-44ae12ab1155 None admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:15.741253 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a3c48b-66da-40a8-b96b-44ae12ab1155 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:15.741536 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a3c48b-66da-40a8-b96b-44ae12ab1155 None admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:15.742362 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a3c48b-66da-40a8-b96b-44ae12ab1155 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:15.744396 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a3c48b-66da-40a8-b96b-44ae12ab1155 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:15.766980 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 15/33] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:14:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 02:14:15.767838 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a439268a-e803-43b3-bcfb-dc61f7a0c4ff None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:15.768133 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a439268a-e803-43b3-bcfb-dc61f7a0c4ff None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:15.768539 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a439268a-e803-43b3-bcfb-dc61f7a0c4ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:15.794812 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-a439268a-e803-43b3-bcfb-dc61f7a0c4ff None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:14:15.796018 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-a439268a-e803-43b3-bcfb-dc61f7a0c4ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:14:15.869700 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 16/34] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:14:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 02:14:15.871151 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a439268a-e803-43b3-bcfb-dc61f7a0c4ff None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:15.974921 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:15.981851 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a439268a-e803-43b3-bcfb-dc61f7a0c4ff None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:16.000462 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 19/35] 10.222.0.66 () {60 vars in 1141 bytes} [Mon Jul 18 02:14:15 2022] POST /identity/v3/projects => generated 301 bytes in 442 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:14:16.001407 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a3c48b-66da-40a8-b96b-44ae12ab1155 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:16.042045 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a9cbba-078e-4510-9204-630a4787aa80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:16.043323 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a9cbba-078e-4510-9204-630a4787aa80 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:16.043814 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a9cbba-078e-4510-9204-630a4787aa80 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:16.044299 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a9cbba-078e-4510-9204-630a4787aa80 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:16.045892 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a9cbba-078e-4510-9204-630a4787aa80 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:16.052446 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a9cbba-078e-4510-9204-630a4787aa80 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:16.054847 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 17/36] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 18 02:14:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:14:16.055978 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a9cbba-078e-4510-9204-630a4787aa80 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:16.166745 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:16.194430 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a9cbba-078e-4510-9204-630a4787aa80 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:16.246301 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41069a5d-b045-44c2-a3af-1ab3f6a14da9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:16.246925 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41069a5d-b045-44c2-a3af-1ab3f6a14da9 None admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:16.247215 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41069a5d-b045-44c2-a3af-1ab3f6a14da9 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:16.247508 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41069a5d-b045-44c2-a3af-1ab3f6a14da9 None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:16.248868 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41069a5d-b045-44c2-a3af-1ab3f6a14da9 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:16.250841 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41069a5d-b045-44c2-a3af-1ab3f6a14da9 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:16.253817 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a3c48b-66da-40a8-b96b-44ae12ab1155 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:16.291272 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd295856-17bd-4305-8334-4908bca1c9c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:16.291964 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd295856-17bd-4305-8334-4908bca1c9c8 None admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:16.292257 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd295856-17bd-4305-8334-4908bca1c9c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:16.292541 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd295856-17bd-4305-8334-4908bca1c9c8 None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:16.293299 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd295856-17bd-4305-8334-4908bca1c9c8 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:16.294891 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd295856-17bd-4305-8334-4908bca1c9c8 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:16.398322 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 20/37] 10.222.0.66 () {60 vars in 1131 bytes} [Mon Jul 18 02:14:16 2022] POST /identity/v3/roles => generated 225 bytes in 397 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:14:16.399289 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd295856-17bd-4305-8334-4908bca1c9c8 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:16.471291 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 18/38] 10.222.0.66 () {60 vars in 1131 bytes} [Mon Jul 18 02:14:16 2022] POST /identity/v3/roles => generated 219 bytes in 416 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:14:16.473552 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fe88f5-e5a6-4634-9ad4-13d23dcbc787 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:16.473552 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fe88f5-e5a6-4634-9ad4-13d23dcbc787 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:16.473806 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fe88f5-e5a6-4634-9ad4-13d23dcbc787 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:16.475206 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 19/39] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:14:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:14:16.476490 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee21949-7a4e-4017-8603-a4d10330cdc2 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:16.476781 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee21949-7a4e-4017-8603-a4d10330cdc2 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:16.477058 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee21949-7a4e-4017-8603-a4d10330cdc2 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:16.478405 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 20/40] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:14:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:14:16.484308 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6bd0ec-0e21-4015-9dcd-2beae6139526 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:16.484673 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6bd0ec-0e21-4015-9dcd-2beae6139526 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:16.485039 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6bd0ec-0e21-4015-9dcd-2beae6139526 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:16.511724 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-bd6bd0ec-0e21-4015-9dcd-2beae6139526 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:14:16.513279 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-bd6bd0ec-0e21-4015-9dcd-2beae6139526 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:14:16.536320 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:16.551249 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:16.557078 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd295856-17bd-4305-8334-4908bca1c9c8 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:16.609585 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 21/41] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:14:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 02:14:16.611778 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8daa45-f971-4722-bcf5-c4f63e53964f None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:16.611778 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8daa45-f971-4722-bcf5-c4f63e53964f None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:16.611914 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8daa45-f971-4722-bcf5-c4f63e53964f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:16.620416 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a7f681-47e6-4915-a130-6e86ce5828bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:16.620976 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a7f681-47e6-4915-a130-6e86ce5828bc None admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:16.621268 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a7f681-47e6-4915-a130-6e86ce5828bc None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:16.621578 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a7f681-47e6-4915-a130-6e86ce5828bc None admin] PATH_INFO: `/v3/domains/default/users/62dd67591e0147fe97b248079359a4de/roles/d4aa3324f0c84887b41d0dc4acf7f30e` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:16.623893 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a7f681-47e6-4915-a130-6e86ce5828bc None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=62dd67591e0147fe97b248079359a4de, role_id=d4aa3324f0c84887b41d0dc4acf7f30e)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:16.625732 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a7f681-47e6-4915-a130-6e86ce5828bc None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:16.634737 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-5a8daa45-f971-4722-bcf5-c4f63e53964f None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:14:16.636245 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-5a8daa45-f971-4722-bcf5-c4f63e53964f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:14:16.713069 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 22/42] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:14:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 02:14:16.713956 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8daa45-f971-4722-bcf5-c4f63e53964f None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:16.759926 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 21/43] 10.222.0.66 () {58 vars in 1361 bytes} [Mon Jul 18 02:14:16 2022] PUT /identity/v3/domains/default/users/62dd67591e0147fe97b248079359a4de/roles/d4aa3324f0c84887b41d0dc4acf7f30e => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:14:16.761424 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a7f681-47e6-4915-a130-6e86ce5828bc None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:16.816113 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:16.834003 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8daa45-f971-4722-bcf5-c4f63e53964f None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:16.874160 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4ce606-f516-4015-ade6-4d77c0ddb489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:16.875144 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4ce606-f516-4015-ade6-4d77c0ddb489 None admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:16.875439 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4ce606-f516-4015-ade6-4d77c0ddb489 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:16.875833 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4ce606-f516-4015-ade6-4d77c0ddb489 None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:16.876489 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4ce606-f516-4015-ade6-4d77c0ddb489 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:16.878053 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4ce606-f516-4015-ade6-4d77c0ddb489 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:16.889080 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a7f681-47e6-4915-a130-6e86ce5828bc None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:16.928873 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b71994e-276f-4186-9de4-c0cea7301195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:16.929678 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b71994e-276f-4186-9de4-c0cea7301195 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:16.929962 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b71994e-276f-4186-9de4-c0cea7301195 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:16.930316 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b71994e-276f-4186-9de4-c0cea7301195 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:16.931479 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b71994e-276f-4186-9de4-c0cea7301195 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:16.932956 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b71994e-276f-4186-9de4-c0cea7301195 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:16.934690 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 22/44] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 18 02:14:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:14:16.940270 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b71994e-276f-4186-9de4-c0cea7301195 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:16.951835 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 23/45] 10.222.0.66 () {60 vars in 1131 bytes} [Mon Jul 18 02:14:16 2022] POST /identity/v3/roles => generated 223 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:14:16.972769 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:17.054772 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b71994e-276f-4186-9de4-c0cea7301195 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:17.093305 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11055a2d-64aa-4e9e-be44-4e7d066371c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:17.094070 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11055a2d-64aa-4e9e-be44-4e7d066371c9 None admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:17.094480 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11055a2d-64aa-4e9e-be44-4e7d066371c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:17.094854 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11055a2d-64aa-4e9e-be44-4e7d066371c9 None admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:17.095684 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11055a2d-64aa-4e9e-be44-4e7d066371c9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:17.097257 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11055a2d-64aa-4e9e-be44-4e7d066371c9 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:17.174846 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 23/46] 10.222.0.66 () {60 vars in 1141 bytes} [Mon Jul 18 02:14:16 2022] POST /identity/v3/projects => generated 312 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 02:14:18.805978 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ca0160-8554-4749-b22f-306a0cf66b34 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:18.806525 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ca0160-8554-4749-b22f-306a0cf66b34 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:18.806608 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ca0160-8554-4749-b22f-306a0cf66b34 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:18.807951 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 24/47] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:14:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:14:18.813306 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8db447-8dd3-4e33-8042-24ab4e5a8da4 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:18.813713 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8db447-8dd3-4e33-8042-24ab4e5a8da4 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:18.813872 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8db447-8dd3-4e33-8042-24ab4e5a8da4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:18.830682 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-fa8db447-8dd3-4e33-8042-24ab4e5a8da4 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:14:18.832784 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-fa8db447-8dd3-4e33-8042-24ab4e5a8da4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:14:18.898464 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 24/48] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:14:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 02:14:18.920999 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ca0160-8554-4749-b22f-306a0cf66b34 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:19.021503 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ca0160-8554-4749-b22f-306a0cf66b34 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:19.054038 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e38a5e-f7c6-4fca-aaf8-34c930610d6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:19.054851 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e38a5e-f7c6-4fca-aaf8-34c930610d6d None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:19.055137 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e38a5e-f7c6-4fca-aaf8-34c930610d6d None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:19.055415 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e38a5e-f7c6-4fca-aaf8-34c930610d6d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:19.065420 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e38a5e-f7c6-4fca-aaf8-34c930610d6d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 02:14:19.073719 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-21e38a5e-f7c6-4fca-aaf8-34c930610d6d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 02:14:19.073719 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:14:19.073719 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:14:19.073719 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:14:19.073719 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:14:19.073719 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:14:19.073719 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:14:19.073719 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:14:19.073719 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:14:19.073719 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:14:19.073719 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:14:19.073719 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:14:19.073719 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 02:14:19.073719 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 02:14:19.073719 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 02:14:19.073719 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 02:14:19.074604 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 02:14:19.074604 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 02:14:19.074604 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 02:14:19.074604 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 02:14:19.074604 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:14:19.074604 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:14:19.074604 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:14:19.074604 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:14:19.074604 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:14:19.074604 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:14:19.074604 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:14:19.074604 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:14:19.074604 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:14:19.074604 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:14:19.074604 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:14:19.074604 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:14:19.075374 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:14:19.075374 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:14:19.075374 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:14:19.075374 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:14:19.075374 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 02:14:19.075374 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 02:14:19.075374 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 02:14:19.075374 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 02:14:19.075374 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 02:14:19.075374 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 02:14:19.075374 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 02:14:19.075374 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:14:19.075374 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 25/49] 10.222.0.66 () {56 vars in 1096 bytes} [Mon Jul 18 02:14:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:14:19.080809 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8db447-8dd3-4e33-8042-24ab4e5a8da4 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:19.091662 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:19.180648 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8db447-8dd3-4e33-8042-24ab4e5a8da4 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:19.203271 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ee8f21-2cea-4252-8a22-08fb5754c622 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:19.203560 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ee8f21-2cea-4252-8a22-08fb5754c622 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:19.203839 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ee8f21-2cea-4252-8a22-08fb5754c622 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:19.205397 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 26/50] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:14:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:14:19.210654 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854c360e-21b6-43d9-b38d-866bd1830ec8 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:19.210947 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854c360e-21b6-43d9-b38d-866bd1830ec8 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:19.211236 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854c360e-21b6-43d9-b38d-866bd1830ec8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:19.216563 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ab939a-fa7a-4870-9030-fe33d262f57f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:19.217245 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ab939a-fa7a-4870-9030-fe33d262f57f None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:19.217536 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ab939a-fa7a-4870-9030-fe33d262f57f None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:19.217817 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ab939a-fa7a-4870-9030-fe33d262f57f None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:19.218656 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ab939a-fa7a-4870-9030-fe33d262f57f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:14:19.218814 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:19.218975 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ab939a-fa7a-4870-9030-fe33d262f57f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:19.220378 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ab939a-fa7a-4870-9030-fe33d262f57f None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:19.228653 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-854c360e-21b6-43d9-b38d-866bd1830ec8 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:14:19.228862 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 25/51] 10.222.0.66 () {56 vars in 1099 bytes} [Mon Jul 18 02:14:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:14:19.229865 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-854c360e-21b6-43d9-b38d-866bd1830ec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:14:19.234449 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ab939a-fa7a-4870-9030-fe33d262f57f None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:19.298972 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 27/52] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:14:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 02:14:19.303365 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:19.323743 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854c360e-21b6-43d9-b38d-866bd1830ec8 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:19.333967 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ab939a-fa7a-4870-9030-fe33d262f57f None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:19.368423 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa8dd5f-4005-47ad-8b69-4e538a559252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:19.369288 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa8dd5f-4005-47ad-8b69-4e538a559252 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:19.369569 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa8dd5f-4005-47ad-8b69-4e538a559252 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:19.369850 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa8dd5f-4005-47ad-8b69-4e538a559252 None admin] PATH_INFO: `/v3/users/62dd67591e0147fe97b248079359a4de` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:19.371120 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa8dd5f-4005-47ad-8b69-4e538a559252 None admin] RBAC: Authorizing `identity:get_user(user_id=62dd67591e0147fe97b248079359a4de)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:19.372586 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa8dd5f-4005-47ad-8b69-4e538a559252 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:19.374422 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 26/53] 10.222.0.66 () {56 vars in 1177 bytes} [Mon Jul 18 02:14:19 2022] GET /identity/v3/users/62dd67591e0147fe97b248079359a4de => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 02:14:19.380515 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa8dd5f-4005-47ad-8b69-4e538a559252 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:19.490367 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa8dd5f-4005-47ad-8b69-4e538a559252 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:19.525175 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269bc02c-0f18-4f91-b170-aa3a558a8e7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:19.525978 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269bc02c-0f18-4f91-b170-aa3a558a8e7b None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:19.526296 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269bc02c-0f18-4f91-b170-aa3a558a8e7b None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:19.526610 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269bc02c-0f18-4f91-b170-aa3a558a8e7b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:19.527907 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269bc02c-0f18-4f91-b170-aa3a558a8e7b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:19.529178 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854c360e-21b6-43d9-b38d-866bd1830ec8 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:19.529399 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269bc02c-0f18-4f91-b170-aa3a558a8e7b None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:19.531138 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 27/54] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 18 02:14:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 02:14:19.537149 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269bc02c-0f18-4f91-b170-aa3a558a8e7b None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:19.546321 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:19.563771 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e5e484-ae01-48e3-80fe-80d389edc642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:19.564570 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e5e484-ae01-48e3-80fe-80d389edc642 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:19.564857 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e5e484-ae01-48e3-80fe-80d389edc642 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:19.565136 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e5e484-ae01-48e3-80fe-80d389edc642 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:19.566329 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e5e484-ae01-48e3-80fe-80d389edc642 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:19.568436 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e5e484-ae01-48e3-80fe-80d389edc642 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:19.570078 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 28/55] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 18 02:14:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 02:14:19.575948 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e5e484-ae01-48e3-80fe-80d389edc642 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:19.578774 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:19.642350 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269bc02c-0f18-4f91-b170-aa3a558a8e7b None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:19.676930 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e5e484-ae01-48e3-80fe-80d389edc642 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:19.677353 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af3e263-72fb-46d4-9525-f8916df75dc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:19.678020 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af3e263-72fb-46d4-9525-f8916df75dc4 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:19.678340 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af3e263-72fb-46d4-9525-f8916df75dc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:19.678626 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af3e263-72fb-46d4-9525-f8916df75dc4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:19.679466 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af3e263-72fb-46d4-9525-f8916df75dc4 None admin] RBAC: Adding query filter params (role.id=d4aa3324f0c84887b41d0dc4acf7f30e, scope.domain.id=default, user.id=62dd67591e0147fe97b248079359a4de) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:14:19.679784 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af3e263-72fb-46d4-9525-f8916df75dc4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:19.681287 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af3e263-72fb-46d4-9525-f8916df75dc4 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:19.687310 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 28/56] 10.222.0.66 () {56 vars in 1322 bytes} [Mon Jul 18 02:14:19 2022] GET /identity/v3/role_assignments?user.id=62dd67591e0147fe97b248079359a4de&scope.domain.id=default&role.id=d4aa3324f0c84887b41d0dc4acf7f30e => generated 530 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 02:14:19.712625 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dc152f-b682-4dfe-ae8d-a17bdc4b4835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:19.713309 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dc152f-b682-4dfe-ae8d-a17bdc4b4835 None admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:19.713598 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dc152f-b682-4dfe-ae8d-a17bdc4b4835 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:19.713879 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dc152f-b682-4dfe-ae8d-a17bdc4b4835 None admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:19.714649 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91dc152f-b682-4dfe-ae8d-a17bdc4b4835 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:19.716218 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91dc152f-b682-4dfe-ae8d-a17bdc4b4835 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:19.797120 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 29/57] 10.222.0.66 () {60 vars in 1140 bytes} [Mon Jul 18 02:14:19 2022] POST /identity/v3/projects => generated 298 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 02:14:21.829465 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb15bef-9338-4770-a784-f4df97d35637 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:21.829972 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb15bef-9338-4770-a784-f4df97d35637 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:21.830101 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb15bef-9338-4770-a784-f4df97d35637 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:21.831464 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 29/58] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:14:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:14:21.837357 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840bbd28-8280-479a-ac69-51daa14c6985 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:21.837786 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840bbd28-8280-479a-ac69-51daa14c6985 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:21.837924 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840bbd28-8280-479a-ac69-51daa14c6985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:21.854982 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-840bbd28-8280-479a-ac69-51daa14c6985 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:14:21.856162 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-840bbd28-8280-479a-ac69-51daa14c6985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:14:21.903902 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:21.921144 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 30/59] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:14:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 02:14:21.945723 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb15bef-9338-4770-a784-f4df97d35637 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:22.015210 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:22.046347 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb15bef-9338-4770-a784-f4df97d35637 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:22.079232 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572a6893-5652-4e8e-9d6e-ec4737e67589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:22.080020 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572a6893-5652-4e8e-9d6e-ec4737e67589 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:22.080309 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572a6893-5652-4e8e-9d6e-ec4737e67589 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:22.080591 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572a6893-5652-4e8e-9d6e-ec4737e67589 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:22.081773 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572a6893-5652-4e8e-9d6e-ec4737e67589 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:22.084084 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572a6893-5652-4e8e-9d6e-ec4737e67589 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:22.085715 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 30/60] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 18 02:14:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:14:22.091231 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840bbd28-8280-479a-ac69-51daa14c6985 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:22.187411 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840bbd28-8280-479a-ac69-51daa14c6985 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:22.220461 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd8a148-a2f7-481b-9866-78828296f8d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:22.221116 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd8a148-a2f7-481b-9866-78828296f8d2 None admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:22.221398 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd8a148-a2f7-481b-9866-78828296f8d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:22.221669 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd8a148-a2f7-481b-9866-78828296f8d2 None admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:22.222414 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd8a148-a2f7-481b-9866-78828296f8d2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:22.223970 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd8a148-a2f7-481b-9866-78828296f8d2 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:22.342862 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 31/61] 10.222.0.66 () {60 vars in 1132 bytes} [Mon Jul 18 02:14:22 2022] POST /identity/v3/users => generated 275 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 02:14:25.461060 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66260399-fc65-4251-89a3-0b9c4e5ee327 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:25.461580 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66260399-fc65-4251-89a3-0b9c4e5ee327 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:25.461654 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66260399-fc65-4251-89a3-0b9c4e5ee327 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:25.463066 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 31/62] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:14:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:14:25.468540 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192cd2d5-dcab-4292-ba38-f8284864f61b None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:25.469373 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192cd2d5-dcab-4292-ba38-f8284864f61b None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:25.469844 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192cd2d5-dcab-4292-ba38-f8284864f61b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:25.495124 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-192cd2d5-dcab-4292-ba38-f8284864f61b None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:14:25.496675 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-192cd2d5-dcab-4292-ba38-f8284864f61b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:14:25.545833 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:25.591216 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 32/63] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:14:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 02:14:25.622848 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66260399-fc65-4251-89a3-0b9c4e5ee327 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:25.635252 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c15557-cf42-470b-9952-13ee4e2d60b7 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:25.635855 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c15557-cf42-470b-9952-13ee4e2d60b7 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:25.636335 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c15557-cf42-470b-9952-13ee4e2d60b7 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:25.637938 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 33/64] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:14:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 02:14:25.643219 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df15efa-fea9-46aa-9a2c-e19271c8064b None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:25.643515 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df15efa-fea9-46aa-9a2c-e19271c8064b None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:25.643804 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df15efa-fea9-46aa-9a2c-e19271c8064b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:25.670665 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-5df15efa-fea9-46aa-9a2c-e19271c8064b None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:14:25.671879 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-5df15efa-fea9-46aa-9a2c-e19271c8064b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:14:25.768662 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66260399-fc65-4251-89a3-0b9c4e5ee327 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:25.768942 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 34/65] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:14:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 02:14:25.771041 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c71c34-1a23-42df-a0f9-fd9f29190a40 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:25.771439 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c71c34-1a23-42df-a0f9-fd9f29190a40 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:25.771943 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c71c34-1a23-42df-a0f9-fd9f29190a40 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:25.773525 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 35/66] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:14:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:14:25.779396 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5df099e-c1a1-471c-8229-2708fa0cf48d None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:25.779940 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5df099e-c1a1-471c-8229-2708fa0cf48d None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:25.780436 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5df099e-c1a1-471c-8229-2708fa0cf48d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:25.808605 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c29fc1b-1c0d-4ec6-a0e4-0779a0bcec67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:25.809419 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c29fc1b-1c0d-4ec6-a0e4-0779a0bcec67 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:25.809708 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c29fc1b-1c0d-4ec6-a0e4-0779a0bcec67 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:25.809997 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c29fc1b-1c0d-4ec6-a0e4-0779a0bcec67 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:25.811922 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-e5df099e-c1a1-471c-8229-2708fa0cf48d None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:14:25.813346 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-e5df099e-c1a1-471c-8229-2708fa0cf48d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:14:25.819213 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:25.824261 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c29fc1b-1c0d-4ec6-a0e4-0779a0bcec67 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 02:14:25.835821 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-6c29fc1b-1c0d-4ec6-a0e4-0779a0bcec67 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 02:14:25.835821 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:14:25.835821 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:14:25.835821 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:14:25.835821 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:14:25.835821 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:14:25.835821 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:14:25.835821 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:14:25.835821 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:14:25.835821 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:14:25.835821 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:14:25.835821 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:14:25.835821 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 02:14:25.835821 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 02:14:25.835821 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 02:14:25.835821 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 02:14:25.836744 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 02:14:25.836744 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 02:14:25.836744 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 02:14:25.836744 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 02:14:25.836744 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:14:25.836744 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:14:25.836744 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:14:25.836744 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:14:25.836744 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:14:25.836744 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:14:25.836744 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:14:25.836744 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:14:25.836744 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:14:25.836744 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:14:25.836744 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:14:25.836744 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:14:25.837937 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:14:25.837937 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:14:25.837937 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:14:25.837937 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:14:25.837937 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 02:14:25.837937 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 02:14:25.837937 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 02:14:25.837937 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 02:14:25.837937 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 02:14:25.837937 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 02:14:25.837937 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 02:14:25.837937 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:14:25.837937 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 32/67] 10.222.0.66 () {56 vars in 1099 bytes} [Mon Jul 18 02:14:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:14:25.837937 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c29fc1b-1c0d-4ec6-a0e4-0779a0bcec67 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:25.892773 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 36/68] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:14:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 02:14:25.894063 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5df099e-c1a1-471c-8229-2708fa0cf48d None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:25.934354 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:25.963206 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c29fc1b-1c0d-4ec6-a0e4-0779a0bcec67 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:26.000675 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc48d0fe-7ee7-4234-a929-fff94278be0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:26.001494 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc48d0fe-7ee7-4234-a929-fff94278be0d None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:26.001774 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc48d0fe-7ee7-4234-a929-fff94278be0d None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:26.002048 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc48d0fe-7ee7-4234-a929-fff94278be0d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:26.008859 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5df099e-c1a1-471c-8229-2708fa0cf48d None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:26.013317 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc48d0fe-7ee7-4234-a929-fff94278be0d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 02:14:26.022491 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-fc48d0fe-7ee7-4234-a929-fff94278be0d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 02:14:26.022491 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:14:26.022491 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:14:26.022491 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:14:26.022491 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:14:26.022491 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:14:26.022491 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:14:26.022491 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:14:26.022491 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:14:26.022491 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:14:26.022491 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:14:26.022491 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:14:26.022491 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 02:14:26.022491 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 02:14:26.022491 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 02:14:26.022491 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 02:14:26.023340 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 02:14:26.023340 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 02:14:26.023340 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 02:14:26.023340 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 02:14:26.023340 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:14:26.023340 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:14:26.023340 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:14:26.023340 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:14:26.023340 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:14:26.023340 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:14:26.023340 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:14:26.023340 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:14:26.023340 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:14:26.023340 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:14:26.023340 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:14:26.023340 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:14:26.024205 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:14:26.024205 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:14:26.024205 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:14:26.024205 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:14:26.024205 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 02:14:26.024205 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 02:14:26.024205 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 02:14:26.024205 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 02:14:26.024205 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 02:14:26.024205 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 02:14:26.024205 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 02:14:26.024205 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:14:26.024205 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 33/69] 10.222.0.66 () {56 vars in 1114 bytes} [Mon Jul 18 02:14:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:14:26.025516 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc48d0fe-7ee7-4234-a929-fff94278be0d None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:26.046856 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77faee31-5d0a-4f06-be78-2530d023d586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:26.047531 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77faee31-5d0a-4f06-be78-2530d023d586 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:26.047815 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77faee31-5d0a-4f06-be78-2530d023d586 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:26.048095 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77faee31-5d0a-4f06-be78-2530d023d586 None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:26.048845 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77faee31-5d0a-4f06-be78-2530d023d586 None admin] RBAC: Adding query filter params (name=member) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:14:26.049154 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77faee31-5d0a-4f06-be78-2530d023d586 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:26.051014 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77faee31-5d0a-4f06-be78-2530d023d586 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:26.061783 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 37/70] 10.222.0.66 () {56 vars in 1101 bytes} [Mon Jul 18 02:14:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:14:26.063220 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3e78c4-0d34-47a6-aa2c-1dbb3c70d8c6 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:26.063526 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3e78c4-0d34-47a6-aa2c-1dbb3c70d8c6 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:26.063937 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3e78c4-0d34-47a6-aa2c-1dbb3c70d8c6 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:26.066581 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 38/71] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:14:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:14:26.071848 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3e78c4-0d34-47a6-aa2c-1dbb3c70d8c6 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:26.092482 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:26.136820 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc48d0fe-7ee7-4234-a929-fff94278be0d None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:26.173464 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac3a3d1-2fbc-40cc-ab49-0f74deaf5c7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:26.174299 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac3a3d1-2fbc-40cc-ab49-0f74deaf5c7c None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:26.174590 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac3a3d1-2fbc-40cc-ab49-0f74deaf5c7c None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:26.174882 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac3a3d1-2fbc-40cc-ab49-0f74deaf5c7c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:26.179659 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:26.186472 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac3a3d1-2fbc-40cc-ab49-0f74deaf5c7c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 02:14:26.193006 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3e78c4-0d34-47a6-aa2c-1dbb3c70d8c6 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:26.195469 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-dac3a3d1-2fbc-40cc-ab49-0f74deaf5c7c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 02:14:26.195469 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:14:26.195469 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:14:26.195469 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:14:26.195469 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:14:26.195469 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:14:26.195469 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:14:26.195469 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:14:26.195469 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:14:26.195469 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:14:26.195469 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:14:26.195469 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:14:26.195469 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 02:14:26.195469 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 02:14:26.195469 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 02:14:26.195469 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 02:14:26.196512 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 02:14:26.196512 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 02:14:26.196512 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 02:14:26.196512 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 02:14:26.196512 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:14:26.196512 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:14:26.196512 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:14:26.196512 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:14:26.196512 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:14:26.196512 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:14:26.196512 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:14:26.196512 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:14:26.196512 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:14:26.196512 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:14:26.196512 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:14:26.196512 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:14:26.197400 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:14:26.197400 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:14:26.197400 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:14:26.197400 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:14:26.197400 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 02:14:26.197400 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 02:14:26.197400 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 02:14:26.197400 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 02:14:26.197400 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 02:14:26.197400 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 02:14:26.197400 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 02:14:26.197400 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:14:26.197400 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 34/72] 10.222.0.66 () {56 vars in 1096 bytes} [Mon Jul 18 02:14:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:14:26.198183 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac3a3d1-2fbc-40cc-ab49-0f74deaf5c7c None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:26.225065 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d76a409-cf9c-416b-95bc-31ba69d0a379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:26.225731 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d76a409-cf9c-416b-95bc-31ba69d0a379 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:26.226011 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d76a409-cf9c-416b-95bc-31ba69d0a379 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:26.226328 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d76a409-cf9c-416b-95bc-31ba69d0a379 None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:26.227074 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d76a409-cf9c-416b-95bc-31ba69d0a379 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:14:26.227384 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d76a409-cf9c-416b-95bc-31ba69d0a379 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:26.228758 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d76a409-cf9c-416b-95bc-31ba69d0a379 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:26.236862 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 39/73] 10.222.0.66 () {56 vars in 1111 bytes} [Mon Jul 18 02:14:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:14:26.238508 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e4a1b4-974a-487d-8be3-8fa339382e94 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:26.238992 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e4a1b4-974a-487d-8be3-8fa339382e94 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:26.239500 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e4a1b4-974a-487d-8be3-8fa339382e94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:26.255950 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-f7e4a1b4-974a-487d-8be3-8fa339382e94 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:14:26.257149 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-f7e4a1b4-974a-487d-8be3-8fa339382e94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:14:26.301774 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac3a3d1-2fbc-40cc-ab49-0f74deaf5c7c None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:26.319595 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:26.322461 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 40/74] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:14:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 02:14:26.323812 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77bd4b9-a5d6-47c1-9898-69ad8eb86c8a None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:26.324106 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77bd4b9-a5d6-47c1-9898-69ad8eb86c8a None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:26.324385 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77bd4b9-a5d6-47c1-9898-69ad8eb86c8a None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:26.329614 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 41/75] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:14:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:14:26.329614 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77bd4b9-a5d6-47c1-9898-69ad8eb86c8a None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:26.357444 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748a1918-5b5f-419d-9334-03555de54ff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:26.358355 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748a1918-5b5f-419d-9334-03555de54ff8 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:26.358658 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748a1918-5b5f-419d-9334-03555de54ff8 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:26.358925 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748a1918-5b5f-419d-9334-03555de54ff8 None admin] PATH_INFO: `/v3/users/8fa31761548948049aa25bd5881382c4` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:26.373754 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748a1918-5b5f-419d-9334-03555de54ff8 None admin] RBAC: Authorizing `identity:get_user(user_id=8fa31761548948049aa25bd5881382c4)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:26.375276 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748a1918-5b5f-419d-9334-03555de54ff8 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:26.376974 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 35/76] 10.222.0.66 () {56 vars in 1177 bytes} [Mon Jul 18 02:14:26 2022] GET /identity/v3/users/8fa31761548948049aa25bd5881382c4 => generated 275 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:14:26.377736 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748a1918-5b5f-419d-9334-03555de54ff8 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:26.411523 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:26.428563 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77bd4b9-a5d6-47c1-9898-69ad8eb86c8a None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:26.462663 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b9f7bd-6385-46f0-9cbf-36bb183144ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:26.463329 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b9f7bd-6385-46f0-9cbf-36bb183144ba None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:26.463615 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b9f7bd-6385-46f0-9cbf-36bb183144ba None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:26.463895 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b9f7bd-6385-46f0-9cbf-36bb183144ba None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:26.464646 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b9f7bd-6385-46f0-9cbf-36bb183144ba None admin] RBAC: Adding query filter params (name=admin) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:14:26.464957 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b9f7bd-6385-46f0-9cbf-36bb183144ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:26.466380 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b9f7bd-6385-46f0-9cbf-36bb183144ba None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:26.475010 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 42/77] 10.222.0.66 () {56 vars in 1099 bytes} [Mon Jul 18 02:14:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:14:26.476187 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0363f629-58f6-40f5-b50f-361ae6d742b4 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:26.476309 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748a1918-5b5f-419d-9334-03555de54ff8 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:26.476672 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0363f629-58f6-40f5-b50f-361ae6d742b4 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:26.476746 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0363f629-58f6-40f5-b50f-361ae6d742b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:26.492320 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-0363f629-58f6-40f5-b50f-361ae6d742b4 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:14:26.493509 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-0363f629-58f6-40f5-b50f-361ae6d742b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:14:26.512561 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2655b80f-8878-49ab-8149-8f2ef1cc9547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:26.513430 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2655b80f-8878-49ab-8149-8f2ef1cc9547 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:26.513740 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2655b80f-8878-49ab-8149-8f2ef1cc9547 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:26.514017 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2655b80f-8878-49ab-8149-8f2ef1cc9547 None admin] PATH_INFO: `/v3/users/8fa31761548948049aa25bd5881382c4` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:26.515228 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2655b80f-8878-49ab-8149-8f2ef1cc9547 None admin] RBAC: Authorizing `identity:get_user(user_id=8fa31761548948049aa25bd5881382c4)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:26.516694 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2655b80f-8878-49ab-8149-8f2ef1cc9547 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:26.518354 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 36/78] 10.222.0.66 () {56 vars in 1177 bytes} [Mon Jul 18 02:14:26 2022] GET /identity/v3/users/8fa31761548948049aa25bd5881382c4 => generated 275 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:14:26.518354 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2655b80f-8878-49ab-8149-8f2ef1cc9547 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:26.563542 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 43/79] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:14:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 02:14:26.564406 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0363f629-58f6-40f5-b50f-361ae6d742b4 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:26.564728 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:26.620893 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2655b80f-8878-49ab-8149-8f2ef1cc9547 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:26.654538 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a44e538-ea06-4118-acb7-7664d551c55d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:26.654700 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:26.655444 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a44e538-ea06-4118-acb7-7664d551c55d None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:26.655728 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a44e538-ea06-4118-acb7-7664d551c55d None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:26.656007 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a44e538-ea06-4118-acb7-7664d551c55d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:26.667160 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a44e538-ea06-4118-acb7-7664d551c55d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 02:14:26.674899 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-6a44e538-ea06-4118-acb7-7664d551c55d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 02:14:26.674899 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:14:26.674899 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:14:26.674899 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:14:26.674899 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:14:26.674899 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:14:26.674899 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:14:26.674899 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:14:26.674899 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:14:26.674899 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:14:26.674899 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:14:26.674899 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:14:26.674899 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 02:14:26.674899 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 02:14:26.674899 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 02:14:26.674899 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 02:14:26.685681 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 02:14:26.685681 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 02:14:26.685681 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 02:14:26.685681 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 02:14:26.685681 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:14:26.685681 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:14:26.685681 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:14:26.685681 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:14:26.685681 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:14:26.685681 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:14:26.685681 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:14:26.685681 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:14:26.685681 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:14:26.685681 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:14:26.685681 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:14:26.685681 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:14:26.686501 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:14:26.686501 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:14:26.686501 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:14:26.686501 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:14:26.686501 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 02:14:26.686501 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 02:14:26.686501 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 02:14:26.686501 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 02:14:26.686501 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 02:14:26.686501 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 02:14:26.686501 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 02:14:26.686501 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:14:26.686501 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 37/80] 10.222.0.66 () {56 vars in 1099 bytes} [Mon Jul 18 02:14:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:14:26.687159 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0363f629-58f6-40f5-b50f-361ae6d742b4 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:26.687227 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a44e538-ea06-4118-acb7-7664d551c55d None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:26.712030 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c22f83-2091-4d55-ae9c-670fc5cc96eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:26.712687 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c22f83-2091-4d55-ae9c-670fc5cc96eb None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:26.712975 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c22f83-2091-4d55-ae9c-670fc5cc96eb None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:26.713257 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c22f83-2091-4d55-ae9c-670fc5cc96eb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:26.777644 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a44e538-ea06-4118-acb7-7664d551c55d None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:26.779046 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c22f83-2091-4d55-ae9c-670fc5cc96eb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:26.779949 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:26.780519 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c22f83-2091-4d55-ae9c-670fc5cc96eb None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:26.811575 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e83819-cdd5-4abe-9406-b3f3b59899a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:26.812459 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e83819-cdd5-4abe-9406-b3f3b59899a3 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:26.813102 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e83819-cdd5-4abe-9406-b3f3b59899a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:26.813661 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e83819-cdd5-4abe-9406-b3f3b59899a3 None admin] PATH_INFO: `/v3/users/62dd67591e0147fe97b248079359a4de` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:26.815064 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e83819-cdd5-4abe-9406-b3f3b59899a3 None admin] RBAC: Authorizing `identity:get_user(user_id=62dd67591e0147fe97b248079359a4de)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:26.816787 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e83819-cdd5-4abe-9406-b3f3b59899a3 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:26.818476 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 38/81] 10.222.0.66 () {56 vars in 1177 bytes} [Mon Jul 18 02:14:26 2022] GET /identity/v3/users/62dd67591e0147fe97b248079359a4de => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:14:26.819242 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e83819-cdd5-4abe-9406-b3f3b59899a3 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:26.855169 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:26.877364 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 44/82] 10.222.0.66 () {58 vars in 1282 bytes} [Mon Jul 18 02:14:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 02:14:26.877364 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c22f83-2091-4d55-ae9c-670fc5cc96eb None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:26.925989 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e83819-cdd5-4abe-9406-b3f3b59899a3 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:26.959443 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ba8ac0-e13a-4e58-ac5e-95041c8d3fb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:26.960089 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ba8ac0-e13a-4e58-ac5e-95041c8d3fb9 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:26.960376 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ba8ac0-e13a-4e58-ac5e-95041c8d3fb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:26.960645 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ba8ac0-e13a-4e58-ac5e-95041c8d3fb9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:26.975901 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c22f83-2091-4d55-ae9c-670fc5cc96eb None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:27.001421 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:27.010215 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610fae82-c2e8-446f-9bd5-6564195218d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:27.011003 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610fae82-c2e8-446f-9bd5-6564195218d2 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:27.011286 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610fae82-c2e8-446f-9bd5-6564195218d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:27.011559 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610fae82-c2e8-446f-9bd5-6564195218d2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:27.013008 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610fae82-c2e8-446f-9bd5-6564195218d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:27.014249 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610fae82-c2e8-446f-9bd5-6564195218d2 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:27.021346 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 45/83] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 18 02:14:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:14:27.021346 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610fae82-c2e8-446f-9bd5-6564195218d2 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:27.029127 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ba8ac0-e13a-4e58-ac5e-95041c8d3fb9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:27.031386 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ba8ac0-e13a-4e58-ac5e-95041c8d3fb9 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:27.062987 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:27.117912 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610fae82-c2e8-446f-9bd5-6564195218d2 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:27.129349 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 39/84] 10.222.0.66 () {58 vars in 1282 bytes} [Mon Jul 18 02:14:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 02:14:27.130331 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ba8ac0-e13a-4e58-ac5e-95041c8d3fb9 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:27.152260 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f622f50a-62ba-47bd-bfb7-35d0ec2fa70c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:27.152937 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f622f50a-62ba-47bd-bfb7-35d0ec2fa70c None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:27.153232 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f622f50a-62ba-47bd-bfb7-35d0ec2fa70c None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:27.153518 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f622f50a-62ba-47bd-bfb7-35d0ec2fa70c None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:27.154305 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f622f50a-62ba-47bd-bfb7-35d0ec2fa70c None admin] RBAC: Adding query filter params (name=member) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:14:27.154619 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f622f50a-62ba-47bd-bfb7-35d0ec2fa70c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:27.156008 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f622f50a-62ba-47bd-bfb7-35d0ec2fa70c None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:27.165071 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 46/85] 10.222.0.66 () {56 vars in 1101 bytes} [Mon Jul 18 02:14:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:14:27.165847 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f622f50a-62ba-47bd-bfb7-35d0ec2fa70c None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:27.201021 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:27.230329 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ba8ac0-e13a-4e58-ac5e-95041c8d3fb9 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:27.264188 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ad9f79-c13f-4253-b706-d108e1f52a26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:27.264838 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ad9f79-c13f-4253-b706-d108e1f52a26 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:27.265127 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ad9f79-c13f-4253-b706-d108e1f52a26 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:27.265409 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ad9f79-c13f-4253-b706-d108e1f52a26 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:27.266429 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f622f50a-62ba-47bd-bfb7-35d0ec2fa70c None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:27.273899 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:27.302052 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1644ba09-9ae7-4f75-b99b-3ae3a185eb7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:27.302934 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1644ba09-9ae7-4f75-b99b-3ae3a185eb7c None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:27.303226 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1644ba09-9ae7-4f75-b99b-3ae3a185eb7c None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:27.303509 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1644ba09-9ae7-4f75-b99b-3ae3a185eb7c None admin] PATH_INFO: `/v3/projects/a5370165a2354ba6aec1bb43138f4838` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:27.304702 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1644ba09-9ae7-4f75-b99b-3ae3a185eb7c None admin] RBAC: Authorizing `identity:get_project(project_id=a5370165a2354ba6aec1bb43138f4838)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:27.306208 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1644ba09-9ae7-4f75-b99b-3ae3a185eb7c None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:27.307925 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 47/86] 10.222.0.66 () {56 vars in 1186 bytes} [Mon Jul 18 02:14:27 2022] GET /identity/v3/projects/a5370165a2354ba6aec1bb43138f4838 => generated 298 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:14:27.308672 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1644ba09-9ae7-4f75-b99b-3ae3a185eb7c None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:27.338726 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ad9f79-c13f-4253-b706-d108e1f52a26 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:27.340214 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ad9f79-c13f-4253-b706-d108e1f52a26 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:27.406946 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:27.407750 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1644ba09-9ae7-4f75-b99b-3ae3a185eb7c None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:27.439743 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 40/87] 10.222.0.66 () {58 vars in 1282 bytes} [Mon Jul 18 02:14:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 02:14:27.440660 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ad9f79-c13f-4253-b706-d108e1f52a26 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:27.445157 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de3b850-9f58-4157-8756-51cf1821de96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:27.445830 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de3b850-9f58-4157-8756-51cf1821de96 None admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:27.446142 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de3b850-9f58-4157-8756-51cf1821de96 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:27.446423 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de3b850-9f58-4157-8756-51cf1821de96 None admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:27.447137 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de3b850-9f58-4157-8756-51cf1821de96 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:27.448587 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de3b850-9f58-4157-8756-51cf1821de96 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:27.515219 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 48/88] 10.222.0.66 () {60 vars in 1132 bytes} [Mon Jul 18 02:14:27 2022] POST /identity/v3/users => generated 289 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:14:27.516200 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de3b850-9f58-4157-8756-51cf1821de96 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:27.548801 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ad9f79-c13f-4253-b706-d108e1f52a26 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:27.553683 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:27.592159 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8a3725-6a3c-4272-a9e7-2dc5dcd93c2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:27.593041 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8a3725-6a3c-4272-a9e7-2dc5dcd93c2d None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:27.593327 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8a3725-6a3c-4272-a9e7-2dc5dcd93c2d None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:27.593604 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8a3725-6a3c-4272-a9e7-2dc5dcd93c2d None admin] PATH_INFO: `/v3/projects/a5370165a2354ba6aec1bb43138f4838` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:27.594890 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8a3725-6a3c-4272-a9e7-2dc5dcd93c2d None admin] RBAC: Authorizing `identity:get_project(project_id=a5370165a2354ba6aec1bb43138f4838)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:27.596404 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8a3725-6a3c-4272-a9e7-2dc5dcd93c2d None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:27.602137 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 41/89] 10.222.0.66 () {56 vars in 1186 bytes} [Mon Jul 18 02:14:27 2022] GET /identity/v3/projects/a5370165a2354ba6aec1bb43138f4838 => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:14:27.602137 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8a3725-6a3c-4272-a9e7-2dc5dcd93c2d None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:27.643283 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de3b850-9f58-4157-8756-51cf1821de96 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:27.652000 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:27.685228 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908ecc64-032d-4274-a165-17daf48a0750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:27.686292 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908ecc64-032d-4274-a165-17daf48a0750 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:27.686292 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908ecc64-032d-4274-a165-17daf48a0750 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:27.686551 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908ecc64-032d-4274-a165-17daf48a0750 None admin] PATH_INFO: `/v3/users/8fa31761548948049aa25bd5881382c4` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:27.689636 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908ecc64-032d-4274-a165-17daf48a0750 None admin] RBAC: Authorizing `identity:get_user(user_id=8fa31761548948049aa25bd5881382c4)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:27.689932 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908ecc64-032d-4274-a165-17daf48a0750 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:27.691994 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 49/90] 10.222.0.66 () {56 vars in 1177 bytes} [Mon Jul 18 02:14:27 2022] GET /identity/v3/users/8fa31761548948049aa25bd5881382c4 => generated 275 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:14:27.692708 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908ecc64-032d-4274-a165-17daf48a0750 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:27.719903 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8a3725-6a3c-4272-a9e7-2dc5dcd93c2d None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:27.762810 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ea9bbd-5cc0-473c-827f-98389f0e728e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:27.764738 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ea9bbd-5cc0-473c-827f-98389f0e728e None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:27.764738 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ea9bbd-5cc0-473c-827f-98389f0e728e None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:27.764738 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ea9bbd-5cc0-473c-827f-98389f0e728e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:27.765002 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ea9bbd-5cc0-473c-827f-98389f0e728e None admin] RBAC: Adding query filter params (role.id=65feabf3b27944a8b01a86c8ac608457, scope.project.id=a5370165a2354ba6aec1bb43138f4838, user.id=8fa31761548948049aa25bd5881382c4) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:14:27.765272 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ea9bbd-5cc0-473c-827f-98389f0e728e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:27.766848 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ea9bbd-5cc0-473c-827f-98389f0e728e None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:27.773518 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 42/91] 10.222.0.66 () {56 vars in 1374 bytes} [Mon Jul 18 02:14:27 2022] GET /identity/v3/role_assignments?user.id=8fa31761548948049aa25bd5881382c4&scope.project.id=a5370165a2354ba6aec1bb43138f4838&role.id=65feabf3b27944a8b01a86c8ac608457 => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:14:27.774458 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ea9bbd-5cc0-473c-827f-98389f0e728e None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:27.818206 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:27.822924 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908ecc64-032d-4274-a165-17daf48a0750 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:27.868924 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfaf0d0-10c2-421a-91e8-8f861ec130f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:27.869818 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfaf0d0-10c2-421a-91e8-8f861ec130f2 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:27.870231 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfaf0d0-10c2-421a-91e8-8f861ec130f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:27.870525 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfaf0d0-10c2-421a-91e8-8f861ec130f2 None admin] PATH_INFO: `/v3/projects/a5370165a2354ba6aec1bb43138f4838` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:27.871938 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfaf0d0-10c2-421a-91e8-8f861ec130f2 None admin] RBAC: Authorizing `identity:get_project(project_id=a5370165a2354ba6aec1bb43138f4838)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:27.873561 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfaf0d0-10c2-421a-91e8-8f861ec130f2 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:27.875444 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 50/92] 10.222.0.66 () {56 vars in 1186 bytes} [Mon Jul 18 02:14:27 2022] GET /identity/v3/projects/a5370165a2354ba6aec1bb43138f4838 => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:14:27.876264 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfaf0d0-10c2-421a-91e8-8f861ec130f2 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:27.909161 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ea9bbd-5cc0-473c-827f-98389f0e728e None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:27.912714 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:27.950491 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9f7ff1-29c8-4372-bb0f-bb5938278c34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:27.951152 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9f7ff1-29c8-4372-bb0f-bb5938278c34 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:27.951437 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9f7ff1-29c8-4372-bb0f-bb5938278c34 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:27.951848 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9f7ff1-29c8-4372-bb0f-bb5938278c34 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:27.952788 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9f7ff1-29c8-4372-bb0f-bb5938278c34 None admin] RBAC: Adding query filter params (role.id=a5b895323e7b4f0db2491abe9ad57ee9, scope.project.id=a5370165a2354ba6aec1bb43138f4838, user.id=8fa31761548948049aa25bd5881382c4) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:14:27.953180 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9f7ff1-29c8-4372-bb0f-bb5938278c34 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:27.954782 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9f7ff1-29c8-4372-bb0f-bb5938278c34 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:27.961505 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 43/93] 10.222.0.66 () {56 vars in 1374 bytes} [Mon Jul 18 02:14:27 2022] GET /identity/v3/role_assignments?user.id=8fa31761548948049aa25bd5881382c4&scope.project.id=a5370165a2354ba6aec1bb43138f4838&role.id=a5b895323e7b4f0db2491abe9ad57ee9 => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:14:27.962589 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9f7ff1-29c8-4372-bb0f-bb5938278c34 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:27.991830 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfaf0d0-10c2-421a-91e8-8f861ec130f2 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:28.032793 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9a887f-e3a8-49ee-bd48-ef61d0d63308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:28.033461 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9a887f-e3a8-49ee-bd48-ef61d0d63308 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:28.033751 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9a887f-e3a8-49ee-bd48-ef61d0d63308 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:28.034037 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9a887f-e3a8-49ee-bd48-ef61d0d63308 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:28.073847 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:28.082976 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9f7ff1-29c8-4372-bb0f-bb5938278c34 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:28.133476 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cec191-c605-450e-a5d1-7a7def4aafd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:28.134182 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cec191-c605-450e-a5d1-7a7def4aafd2 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:28.134471 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cec191-c605-450e-a5d1-7a7def4aafd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:28.134747 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cec191-c605-450e-a5d1-7a7def4aafd2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:28.135597 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cec191-c605-450e-a5d1-7a7def4aafd2 None admin] RBAC: Adding query filter params (role.id=d4aa3324f0c84887b41d0dc4acf7f30e, scope.project.id=a5370165a2354ba6aec1bb43138f4838, user.id=62dd67591e0147fe97b248079359a4de) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:14:28.135912 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cec191-c605-450e-a5d1-7a7def4aafd2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:28.137416 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cec191-c605-450e-a5d1-7a7def4aafd2 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:28.145024 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 44/94] 10.222.0.66 () {56 vars in 1374 bytes} [Mon Jul 18 02:14:27 2022] GET /identity/v3/role_assignments?user.id=62dd67591e0147fe97b248079359a4de&scope.project.id=a5370165a2354ba6aec1bb43138f4838&role.id=d4aa3324f0c84887b41d0dc4acf7f30e => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:14:28.148713 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9a887f-e3a8-49ee-bd48-ef61d0d63308 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:28.150283 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9a887f-e3a8-49ee-bd48-ef61d0d63308 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:28.268237 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 51/95] 10.222.0.66 () {58 vars in 1282 bytes} [Mon Jul 18 02:14:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 02:14:28.274245 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cec191-c605-450e-a5d1-7a7def4aafd2 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:28.350272 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:28.462050 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cec191-c605-450e-a5d1-7a7def4aafd2 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:28.498529 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0819ed-05c8-4bb1-b7f0-1abf426481fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:28.499409 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0819ed-05c8-4bb1-b7f0-1abf426481fa None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:28.499694 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0819ed-05c8-4bb1-b7f0-1abf426481fa None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:28.499972 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0819ed-05c8-4bb1-b7f0-1abf426481fa None admin] PATH_INFO: `/v3/projects/9b769253e8b5410b9c5fabbea39b7078` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:28.501189 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0819ed-05c8-4bb1-b7f0-1abf426481fa None admin] RBAC: Authorizing `identity:get_project(project_id=9b769253e8b5410b9c5fabbea39b7078)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:28.502684 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0819ed-05c8-4bb1-b7f0-1abf426481fa None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:28.504794 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 45/96] 10.222.0.66 () {56 vars in 1186 bytes} [Mon Jul 18 02:14:28 2022] GET /identity/v3/projects/9b769253e8b5410b9c5fabbea39b7078 => generated 312 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:14:28.510651 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9a887f-e3a8-49ee-bd48-ef61d0d63308 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:28.608580 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:28.653492 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9a887f-e3a8-49ee-bd48-ef61d0d63308 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:28.689705 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446df213-f0dc-4052-9afd-1c9e4984f845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:28.690409 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446df213-f0dc-4052-9afd-1c9e4984f845 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:28.690775 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446df213-f0dc-4052-9afd-1c9e4984f845 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:28.691067 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446df213-f0dc-4052-9afd-1c9e4984f845 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:28.692015 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446df213-f0dc-4052-9afd-1c9e4984f845 None admin] RBAC: Adding query filter params (role.id=65feabf3b27944a8b01a86c8ac608457, scope.project.id=9b769253e8b5410b9c5fabbea39b7078, user.id=8fa31761548948049aa25bd5881382c4) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:14:28.692335 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446df213-f0dc-4052-9afd-1c9e4984f845 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:28.693802 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446df213-f0dc-4052-9afd-1c9e4984f845 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:28.702624 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 52/97] 10.222.0.66 () {56 vars in 1374 bytes} [Mon Jul 18 02:14:28 2022] GET /identity/v3/role_assignments?user.id=8fa31761548948049aa25bd5881382c4&scope.project.id=9b769253e8b5410b9c5fabbea39b7078&role.id=65feabf3b27944a8b01a86c8ac608457 => generated 259 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:14:30.531155 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8055a6-4e2f-43eb-8a70-fdfaf488ee27 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:30.531862 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8055a6-4e2f-43eb-8a70-fdfaf488ee27 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:30.531862 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8055a6-4e2f-43eb-8a70-fdfaf488ee27 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:30.533107 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 46/98] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:14:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:14:30.544097 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25edfa6-812f-4cdb-964e-e4ca3a48474a None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:30.544633 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25edfa6-812f-4cdb-964e-e4ca3a48474a None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:30.544788 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25edfa6-812f-4cdb-964e-e4ca3a48474a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:30.570135 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-c25edfa6-812f-4cdb-964e-e4ca3a48474a None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:14:30.570135 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-c25edfa6-812f-4cdb-964e-e4ca3a48474a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:14:30.574534 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99da71b5-898e-474f-801d-ee0544f8e879 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:30.575085 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99da71b5-898e-474f-801d-ee0544f8e879 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:30.575564 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99da71b5-898e-474f-801d-ee0544f8e879 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:30.577630 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 47/99] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:14:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:14:30.583698 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3cce3d-091d-4a2c-92d1-1ea4abed9040 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:30.584658 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3cce3d-091d-4a2c-92d1-1ea4abed9040 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:30.585166 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3cce3d-091d-4a2c-92d1-1ea4abed9040 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:30.605740 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-2c3cce3d-091d-4a2c-92d1-1ea4abed9040 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:14:30.607303 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-2c3cce3d-091d-4a2c-92d1-1ea4abed9040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:14:30.691638 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 53/100] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:14:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 02:14:30.718181 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 48/101] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:14:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 02:14:30.727221 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25edfa6-812f-4cdb-964e-e4ca3a48474a None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:30.757393 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3cce3d-091d-4a2c-92d1-1ea4abed9040 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:30.823894 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:30.824790 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:30.874382 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25edfa6-812f-4cdb-964e-e4ca3a48474a None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:30.928847 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b952b3c1-76a1-4f33-a25d-1dfc9458c32a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:30.930263 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b952b3c1-76a1-4f33-a25d-1dfc9458c32a None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:30.930702 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b952b3c1-76a1-4f33-a25d-1dfc9458c32a None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:30.931158 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b952b3c1-76a1-4f33-a25d-1dfc9458c32a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:30.932533 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b952b3c1-76a1-4f33-a25d-1dfc9458c32a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:30.934193 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b952b3c1-76a1-4f33-a25d-1dfc9458c32a None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:30.936430 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 54/102] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 18 02:14:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:14:30.938033 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d32ede-354c-4ea9-b6be-d68c3253f62b None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:30.938515 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d32ede-354c-4ea9-b6be-d68c3253f62b None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:30.938943 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d32ede-354c-4ea9-b6be-d68c3253f62b None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:30.940364 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 55/103] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:14:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:14:30.942216 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d32ede-354c-4ea9-b6be-d68c3253f62b None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:30.981681 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3cce3d-091d-4a2c-92d1-1ea4abed9040 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:31.046395 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4f1b13-df9c-4383-bbce-764eebf603b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:31.047565 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4f1b13-df9c-4383-bbce-764eebf603b8 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:31.048077 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4f1b13-df9c-4383-bbce-764eebf603b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:31.048553 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4f1b13-df9c-4383-bbce-764eebf603b8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:31.060083 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4f1b13-df9c-4383-bbce-764eebf603b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 02:14:31.068433 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-4c4f1b13-df9c-4383-bbce-764eebf603b8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 02:14:31.068433 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:14:31.068433 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:14:31.068433 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:14:31.068433 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:14:31.068433 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:14:31.068433 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:14:31.068433 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:14:31.068433 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:14:31.068433 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:14:31.068433 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:14:31.068433 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:14:31.068433 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 02:14:31.068433 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 02:14:31.068433 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 02:14:31.068433 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 02:14:31.069374 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 02:14:31.069374 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 02:14:31.069374 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 02:14:31.069374 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 02:14:31.069374 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:14:31.069374 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:14:31.069374 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:14:31.069374 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:14:31.069374 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:14:31.069374 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:14:31.069374 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:14:31.069374 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:14:31.069374 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:14:31.069374 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:14:31.069374 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:14:31.069374 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:14:31.070193 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:14:31.070193 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:14:31.070193 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:14:31.070193 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:14:31.070193 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 02:14:31.070193 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 02:14:31.070193 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 02:14:31.070193 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 02:14:31.070193 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 02:14:31.070193 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 02:14:31.070193 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 02:14:31.070193 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:14:31.072595 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 49/104] 10.222.0.66 () {56 vars in 1099 bytes} [Mon Jul 18 02:14:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 02:14:31.074122 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b088a0fe-9a98-4559-9a5e-152f58c356af None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:31.074619 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b088a0fe-9a98-4559-9a5e-152f58c356af None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:31.075175 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b088a0fe-9a98-4559-9a5e-152f58c356af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:31.092403 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-b088a0fe-9a98-4559-9a5e-152f58c356af None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:14:31.093912 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-b088a0fe-9a98-4559-9a5e-152f58c356af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:14:31.101400 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d32ede-354c-4ea9-b6be-d68c3253f62b None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:31.158563 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:31.164474 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13a5dcb-52cc-4c22-adf0-249dcbda6feb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:31.165243 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13a5dcb-52cc-4c22-adf0-249dcbda6feb None admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:31.165537 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13a5dcb-52cc-4c22-adf0-249dcbda6feb None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:31.165819 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13a5dcb-52cc-4c22-adf0-249dcbda6feb None admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:31.166619 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13a5dcb-52cc-4c22-adf0-249dcbda6feb None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:31.167399 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 50/105] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:14:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 02:14:31.168298 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13a5dcb-52cc-4c22-adf0-249dcbda6feb None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:31.168840 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b088a0fe-9a98-4559-9a5e-152f58c356af None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:31.194487 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:31.275098 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b088a0fe-9a98-4559-9a5e-152f58c356af None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:31.313128 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 56/106] 10.222.0.66 () {60 vars in 1141 bytes} [Mon Jul 18 02:14:30 2022] POST /identity/v3/projects => generated 302 bytes in 371 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 02:14:31.315701 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13a5dcb-52cc-4c22-adf0-249dcbda6feb None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:31.321388 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f482eca0-e35a-4a4f-b1a4-d035fccef426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:31.322553 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f482eca0-e35a-4a4f-b1a4-d035fccef426 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:31.323522 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f482eca0-e35a-4a4f-b1a4-d035fccef426 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:31.324777 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f482eca0-e35a-4a4f-b1a4-d035fccef426 None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:31.325891 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f482eca0-e35a-4a4f-b1a4-d035fccef426 None admin] RBAC: Adding query filter params (name=member) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:14:31.329056 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f482eca0-e35a-4a4f-b1a4-d035fccef426 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:31.330894 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f482eca0-e35a-4a4f-b1a4-d035fccef426 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:31.349253 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 51/107] 10.222.0.66 () {56 vars in 1101 bytes} [Mon Jul 18 02:14:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:14:31.351317 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4afcc5-2e74-4fc8-862e-7443f535506a None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:31.351864 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4afcc5-2e74-4fc8-862e-7443f535506a None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:31.352324 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4afcc5-2e74-4fc8-862e-7443f535506a None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:31.353832 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 52/108] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:14:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:14:31.355282 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d360295-79a4-444a-a4cf-576b18cff74e None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:31.355831 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d360295-79a4-444a-a4cf-576b18cff74e None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:31.356296 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d360295-79a4-444a-a4cf-576b18cff74e None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:31.358106 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 53/109] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:14:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:14:31.359549 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a314973-78f8-4aa0-b6cd-65e47f5fb0b7 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:31.359841 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a314973-78f8-4aa0-b6cd-65e47f5fb0b7 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:31.360539 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a314973-78f8-4aa0-b6cd-65e47f5fb0b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:31.509750 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-5a314973-78f8-4aa0-b6cd-65e47f5fb0b7 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:14:31.511776 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-5a314973-78f8-4aa0-b6cd-65e47f5fb0b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:14:31.579090 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13a5dcb-52cc-4c22-adf0-249dcbda6feb None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:31.618333 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 54/110] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:14:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 02:14:31.618333 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a314973-78f8-4aa0-b6cd-65e47f5fb0b7 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:31.628634 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:31.634118 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e5b49c-68bb-45d5-bb53-912cc701132f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:31.634972 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e5b49c-68bb-45d5-bb53-912cc701132f None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:31.635261 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e5b49c-68bb-45d5-bb53-912cc701132f None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:31.635548 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e5b49c-68bb-45d5-bb53-912cc701132f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:31.648127 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e5b49c-68bb-45d5-bb53-912cc701132f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 02:14:31.656337 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-f6e5b49c-68bb-45d5-bb53-912cc701132f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 02:14:31.656337 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:14:31.656337 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:14:31.656337 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:14:31.656337 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:14:31.656337 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:14:31.656337 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:14:31.656337 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:14:31.656337 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:14:31.656337 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:14:31.656337 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:14:31.656337 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:14:31.656337 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 02:14:31.656337 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 02:14:31.656337 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 02:14:31.656337 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 02:14:31.657325 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 02:14:31.657325 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 02:14:31.657325 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 02:14:31.657325 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 02:14:31.657325 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:14:31.657325 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:14:31.657325 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:14:31.657325 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:14:31.657325 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:14:31.657325 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:14:31.657325 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:14:31.657325 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:14:31.657325 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:14:31.657325 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:14:31.657325 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:14:31.657325 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:14:31.658146 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:14:31.658146 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:14:31.658146 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:14:31.658146 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:14:31.658146 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 02:14:31.658146 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 02:14:31.658146 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 02:14:31.658146 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 02:14:31.658146 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 02:14:31.658146 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 02:14:31.658146 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 02:14:31.658146 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:14:31.658146 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 57/111] 10.222.0.66 () {56 vars in 1114 bytes} [Mon Jul 18 02:14:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 343 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:14:31.658822 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a14589c-cb9a-4973-81cc-537627914756 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:31.659115 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a14589c-cb9a-4973-81cc-537627914756 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:31.659577 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a14589c-cb9a-4973-81cc-537627914756 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:31.680290 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-1a14589c-cb9a-4973-81cc-537627914756 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:14:31.681530 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-1a14589c-cb9a-4973-81cc-537627914756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:14:31.718966 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:31.723961 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a314973-78f8-4aa0-b6cd-65e47f5fb0b7 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:31.755748 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 58/112] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:14:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 02:14:31.757128 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d2a9e3-0d21-46e1-a41d-57a591268db6 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:31.757426 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d2a9e3-0d21-46e1-a41d-57a591268db6 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:31.757710 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d2a9e3-0d21-46e1-a41d-57a591268db6 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:31.762220 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 59/113] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:14:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:14:31.762220 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d2a9e3-0d21-46e1-a41d-57a591268db6 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:31.782805 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8771e3d-4afb-4f14-8c7e-71f3927b0316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:31.783711 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8771e3d-4afb-4f14-8c7e-71f3927b0316 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:31.784000 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8771e3d-4afb-4f14-8c7e-71f3927b0316 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:31.784921 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8771e3d-4afb-4f14-8c7e-71f3927b0316 None admin] PATH_INFO: `/v3/users/8fa31761548948049aa25bd5881382c4` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:31.786194 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8771e3d-4afb-4f14-8c7e-71f3927b0316 None admin] RBAC: Authorizing `identity:get_user(user_id=8fa31761548948049aa25bd5881382c4)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:31.787777 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8771e3d-4afb-4f14-8c7e-71f3927b0316 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:31.789469 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 55/114] 10.222.0.66 () {56 vars in 1177 bytes} [Mon Jul 18 02:14:31 2022] GET /identity/v3/users/8fa31761548948049aa25bd5881382c4 => generated 275 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:14:31.791341 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8771e3d-4afb-4f14-8c7e-71f3927b0316 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:31.876957 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d2a9e3-0d21-46e1-a41d-57a591268db6 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:31.879569 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:31.900361 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8771e3d-4afb-4f14-8c7e-71f3927b0316 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:31.916085 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20ad710-68b3-44ff-b762-9e6618ba57c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:31.916889 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20ad710-68b3-44ff-b762-9e6618ba57c6 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:31.917183 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20ad710-68b3-44ff-b762-9e6618ba57c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:31.917463 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20ad710-68b3-44ff-b762-9e6618ba57c6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:31.928583 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20ad710-68b3-44ff-b762-9e6618ba57c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 02:14:31.936439 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f79666-75a1-45ae-b728-b286ff2784ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:31.936696 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-e20ad710-68b3-44ff-b762-9e6618ba57c6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 02:14:31.936696 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:14:31.936696 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:14:31.936696 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:14:31.936696 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:14:31.936696 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:14:31.936696 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:14:31.936696 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:14:31.936696 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:14:31.936696 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:14:31.936696 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:14:31.936696 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:14:31.936696 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 02:14:31.936696 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 02:14:31.936696 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 02:14:31.936696 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 02:14:31.937632 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 02:14:31.937632 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 02:14:31.937632 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 02:14:31.937632 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 02:14:31.937632 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:14:31.937632 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:14:31.937632 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:14:31.937632 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:14:31.937632 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:14:31.937632 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:14:31.937632 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:14:31.937632 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:14:31.937632 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:14:31.937632 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:14:31.937632 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:14:31.937632 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:14:31.938764 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:14:31.938764 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:14:31.938764 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:14:31.938764 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:14:31.938764 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 02:14:31.938764 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 02:14:31.938764 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 02:14:31.938764 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 02:14:31.938764 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 02:14:31.938764 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 02:14:31.938764 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 02:14:31.938764 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:14:31.939592 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f79666-75a1-45ae-b728-b286ff2784ba None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:31.939592 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f79666-75a1-45ae-b728-b286ff2784ba None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:31.939592 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f79666-75a1-45ae-b728-b286ff2784ba None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:31.939774 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 60/115] 10.222.0.66 () {56 vars in 1099 bytes} [Mon Jul 18 02:14:31 2022] GET /identity/v3/roles/reader => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:14:31.939774 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57674234-3e51-4943-8142-bda69f8abc01 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:31.939774 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57674234-3e51-4943-8142-bda69f8abc01 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:31.939774 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57674234-3e51-4943-8142-bda69f8abc01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:31.942896 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f79666-75a1-45ae-b728-b286ff2784ba None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:14:31.943379 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f79666-75a1-45ae-b728-b286ff2784ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:31.945025 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f79666-75a1-45ae-b728-b286ff2784ba None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:31.953688 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 56/116] 10.222.0.66 () {56 vars in 1111 bytes} [Mon Jul 18 02:14:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:14:31.954525 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f79666-75a1-45ae-b728-b286ff2784ba None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:31.962590 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-57674234-3e51-4943-8142-bda69f8abc01 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:14:31.963876 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-57674234-3e51-4943-8142-bda69f8abc01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:14:31.989937 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:32.050628 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 61/117] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:14:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 02:14:32.051485 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57674234-3e51-4943-8142-bda69f8abc01 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:32.063564 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f79666-75a1-45ae-b728-b286ff2784ba None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:32.166213 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc453fcc-c24e-4227-9869-a503da4acc14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:32.166213 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc453fcc-c24e-4227-9869-a503da4acc14 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:32.166213 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc453fcc-c24e-4227-9869-a503da4acc14 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:32.166213 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc453fcc-c24e-4227-9869-a503da4acc14 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:32.166663 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc453fcc-c24e-4227-9869-a503da4acc14 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 02:14:32.166663 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-cc453fcc-c24e-4227-9869-a503da4acc14 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 02:14:32.166663 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:14:32.166663 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:14:32.166663 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:14:32.166663 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:14:32.166663 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:14:32.166663 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:14:32.166663 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:14:32.166663 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:14:32.166663 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:14:32.166663 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:14:32.166663 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:14:32.166663 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 02:14:32.167575 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 02:14:32.167575 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 02:14:32.167575 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 02:14:32.167575 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 02:14:32.167575 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 02:14:32.167575 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 02:14:32.167575 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 02:14:32.167575 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:14:32.167575 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:14:32.167575 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:14:32.167575 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:14:32.167575 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:14:32.167575 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:14:32.167575 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:14:32.167575 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:14:32.167575 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:14:32.167575 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:14:32.168478 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:14:32.168478 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:14:32.168478 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:14:32.168478 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:14:32.168478 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:14:32.168478 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:14:32.168478 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 02:14:32.168478 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 02:14:32.168478 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 02:14:32.168478 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 02:14:32.168478 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 02:14:32.168478 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 02:14:32.168478 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 02:14:32.168478 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:14:32.168478 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 57/118] 10.222.0.66 () {56 vars in 1099 bytes} [Mon Jul 18 02:14:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:14:32.169398 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc453fcc-c24e-4227-9869-a503da4acc14 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:32.169398 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:32.171797 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57674234-3e51-4943-8142-bda69f8abc01 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:32.210153 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae379f0e-92e8-4d61-a2ad-c789d0606995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:32.212090 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae379f0e-92e8-4d61-a2ad-c789d0606995 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:32.212388 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae379f0e-92e8-4d61-a2ad-c789d0606995 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:32.212670 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae379f0e-92e8-4d61-a2ad-c789d0606995 None admin] PATH_INFO: `/v3/projects/a5370165a2354ba6aec1bb43138f4838` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:32.213857 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae379f0e-92e8-4d61-a2ad-c789d0606995 None admin] RBAC: Authorizing `identity:get_project(project_id=a5370165a2354ba6aec1bb43138f4838)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:32.215361 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae379f0e-92e8-4d61-a2ad-c789d0606995 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:32.217251 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 62/119] 10.222.0.66 () {56 vars in 1186 bytes} [Mon Jul 18 02:14:32 2022] GET /identity/v3/projects/a5370165a2354ba6aec1bb43138f4838 => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:14:32.222679 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae379f0e-92e8-4d61-a2ad-c789d0606995 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:32.247226 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc453fcc-c24e-4227-9869-a503da4acc14 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:32.255379 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:32.283180 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c355135-4ba6-4149-b7eb-a99f10616526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:32.283968 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c355135-4ba6-4149-b7eb-a99f10616526 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:32.284502 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c355135-4ba6-4149-b7eb-a99f10616526 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:32.284975 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c355135-4ba6-4149-b7eb-a99f10616526 None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:32.285805 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c355135-4ba6-4149-b7eb-a99f10616526 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:14:32.286154 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c355135-4ba6-4149-b7eb-a99f10616526 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:32.287565 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c355135-4ba6-4149-b7eb-a99f10616526 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:32.295735 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 58/120] 10.222.0.66 () {56 vars in 1101 bytes} [Mon Jul 18 02:14:32 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:14:32.296855 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c355135-4ba6-4149-b7eb-a99f10616526 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:32.338117 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae379f0e-92e8-4d61-a2ad-c789d0606995 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:32.362076 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:32.375059 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377a59da-3355-4df6-911b-4856477fb6de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:32.376029 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377a59da-3355-4df6-911b-4856477fb6de None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:32.376335 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377a59da-3355-4df6-911b-4856477fb6de None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:32.376633 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377a59da-3355-4df6-911b-4856477fb6de None admin] PATH_INFO: `/v3/users/8fa31761548948049aa25bd5881382c4` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:32.377881 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377a59da-3355-4df6-911b-4856477fb6de None admin] RBAC: Authorizing `identity:get_user(user_id=8fa31761548948049aa25bd5881382c4)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:32.379578 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377a59da-3355-4df6-911b-4856477fb6de None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:32.381312 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 63/121] 10.222.0.66 () {56 vars in 1177 bytes} [Mon Jul 18 02:14:32 2022] GET /identity/v3/users/8fa31761548948049aa25bd5881382c4 => generated 275 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:14:32.385863 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377a59da-3355-4df6-911b-4856477fb6de None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:32.409753 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c355135-4ba6-4149-b7eb-a99f10616526 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:32.444805 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8df6136-c963-4777-b484-3d8bb5eb5440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:32.445582 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8df6136-c963-4777-b484-3d8bb5eb5440 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:32.445865 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8df6136-c963-4777-b484-3d8bb5eb5440 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:32.446172 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8df6136-c963-4777-b484-3d8bb5eb5440 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:32.457429 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8df6136-c963-4777-b484-3d8bb5eb5440 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 02:14:32.465412 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-e8df6136-c963-4777-b484-3d8bb5eb5440 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 02:14:32.465412 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:14:32.465412 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:14:32.465412 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:14:32.465412 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:14:32.465412 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:14:32.465412 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:14:32.465412 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:14:32.465412 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:14:32.465412 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:14:32.465412 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:14:32.465412 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:14:32.465412 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 02:14:32.465412 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 02:14:32.465412 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 02:14:32.465412 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 02:14:32.466484 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 02:14:32.466484 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 02:14:32.466484 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 02:14:32.466484 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 02:14:32.466484 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:14:32.466484 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:14:32.466484 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:14:32.466484 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:14:32.466484 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:14:32.466484 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:14:32.466484 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:14:32.466484 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:14:32.466484 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:14:32.466484 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:14:32.466484 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:14:32.466484 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:14:32.467303 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:14:32.467303 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:14:32.467303 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:14:32.467303 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:14:32.467303 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 02:14:32.467303 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 02:14:32.467303 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 02:14:32.467303 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 02:14:32.467303 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 02:14:32.467303 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 02:14:32.467303 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 02:14:32.467303 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:14:32.467303 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 59/122] 10.222.0.66 () {56 vars in 1096 bytes} [Mon Jul 18 02:14:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:14:32.467997 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8df6136-c963-4777-b484-3d8bb5eb5440 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:32.495179 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:32.502420 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377a59da-3355-4df6-911b-4856477fb6de None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:32.539635 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be88e32f-b2a7-45f8-a180-ae64f5972ea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:32.540306 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be88e32f-b2a7-45f8-a180-ae64f5972ea1 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:32.540596 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be88e32f-b2a7-45f8-a180-ae64f5972ea1 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:32.540882 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be88e32f-b2a7-45f8-a180-ae64f5972ea1 None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:32.541635 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be88e32f-b2a7-45f8-a180-ae64f5972ea1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:14:32.541947 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be88e32f-b2a7-45f8-a180-ae64f5972ea1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:32.543423 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be88e32f-b2a7-45f8-a180-ae64f5972ea1 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:32.557049 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 64/123] 10.222.0.66 () {56 vars in 1101 bytes} [Mon Jul 18 02:14:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:14:32.557049 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be88e32f-b2a7-45f8-a180-ae64f5972ea1 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:32.571878 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8df6136-c963-4777-b484-3d8bb5eb5440 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:32.596216 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:32.609584 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c22522-6637-4ac7-ac09-41163613f1f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:32.613957 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c22522-6637-4ac7-ac09-41163613f1f3 None admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:32.613957 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c22522-6637-4ac7-ac09-41163613f1f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:32.613957 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c22522-6637-4ac7-ac09-41163613f1f3 None admin] PATH_INFO: `/v3/projects/a5370165a2354ba6aec1bb43138f4838/users/8fa31761548948049aa25bd5881382c4/roles/65feabf3b27944a8b01a86c8ac608457` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:32.613957 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c22522-6637-4ac7-ac09-41163613f1f3 None admin] RBAC: Authorizing `identity:create_grant(project_id=a5370165a2354ba6aec1bb43138f4838, user_id=8fa31761548948049aa25bd5881382c4, role_id=65feabf3b27944a8b01a86c8ac608457)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:32.614765 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c22522-6637-4ac7-ac09-41163613f1f3 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:32.669576 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be88e32f-b2a7-45f8-a180-ae64f5972ea1 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:32.681405 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 60/124] 10.222.0.66 () {58 vars in 1439 bytes} [Mon Jul 18 02:14:32 2022] PUT /identity/v3/projects/a5370165a2354ba6aec1bb43138f4838/users/8fa31761548948049aa25bd5881382c4/roles/65feabf3b27944a8b01a86c8ac608457 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:14:32.689879 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c22522-6637-4ac7-ac09-41163613f1f3 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:32.715943 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c184ebd8-ee2d-4c9e-b80a-86bb990f29cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:32.717288 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c184ebd8-ee2d-4c9e-b80a-86bb990f29cf None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:32.717845 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c184ebd8-ee2d-4c9e-b80a-86bb990f29cf None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:32.718414 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c184ebd8-ee2d-4c9e-b80a-86bb990f29cf None admin] PATH_INFO: `/v3/users/20fb40a9faec4e74bcb9275b58037782` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:32.737716 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c184ebd8-ee2d-4c9e-b80a-86bb990f29cf None admin] RBAC: Authorizing `identity:get_user(user_id=20fb40a9faec4e74bcb9275b58037782)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:32.739857 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c184ebd8-ee2d-4c9e-b80a-86bb990f29cf None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:32.741931 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 65/125] 10.222.0.66 () {56 vars in 1177 bytes} [Mon Jul 18 02:14:32 2022] GET /identity/v3/users/20fb40a9faec4e74bcb9275b58037782 => generated 289 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:14:32.743000 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c184ebd8-ee2d-4c9e-b80a-86bb990f29cf None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:32.764671 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:32.829910 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c22522-6637-4ac7-ac09-41163613f1f3 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:32.860012 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c184ebd8-ee2d-4c9e-b80a-86bb990f29cf None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:32.882424 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e7d3ee-611f-4c18-8c16-91595a2afed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:32.883567 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e7d3ee-611f-4c18-8c16-91595a2afed5 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:32.884059 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e7d3ee-611f-4c18-8c16-91595a2afed5 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:32.884442 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e7d3ee-611f-4c18-8c16-91595a2afed5 None admin] PATH_INFO: `/v3/projects/a5370165a2354ba6aec1bb43138f4838` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:32.886444 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e7d3ee-611f-4c18-8c16-91595a2afed5 None admin] RBAC: Authorizing `identity:get_project(project_id=a5370165a2354ba6aec1bb43138f4838)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:32.888131 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e7d3ee-611f-4c18-8c16-91595a2afed5 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:32.890455 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 61/126] 10.222.0.66 () {56 vars in 1186 bytes} [Mon Jul 18 02:14:32 2022] GET /identity/v3/projects/a5370165a2354ba6aec1bb43138f4838 => generated 298 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:14:32.891451 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e7d3ee-611f-4c18-8c16-91595a2afed5 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:32.903936 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb82c0df-f7e6-4d62-95c2-a780067aefea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:32.904623 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb82c0df-f7e6-4d62-95c2-a780067aefea None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:32.905655 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb82c0df-f7e6-4d62-95c2-a780067aefea None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:32.905945 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb82c0df-f7e6-4d62-95c2-a780067aefea None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:32.906885 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb82c0df-f7e6-4d62-95c2-a780067aefea None admin] RBAC: Adding query filter params (name=admin) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:14:32.907219 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb82c0df-f7e6-4d62-95c2-a780067aefea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:32.908599 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb82c0df-f7e6-4d62-95c2-a780067aefea None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:32.920606 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 66/127] 10.222.0.66 () {56 vars in 1099 bytes} [Mon Jul 18 02:14:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:14:32.922395 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb82c0df-f7e6-4d62-95c2-a780067aefea None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:32.959155 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:33.014469 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e7d3ee-611f-4c18-8c16-91595a2afed5 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:33.023601 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:33.052959 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb82c0df-f7e6-4d62-95c2-a780067aefea None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:33.061670 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eb04e8-4abf-4adc-a2d1-2ae71d5730b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:33.063933 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eb04e8-4abf-4adc-a2d1-2ae71d5730b6 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:33.064712 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eb04e8-4abf-4adc-a2d1-2ae71d5730b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:33.065206 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eb04e8-4abf-4adc-a2d1-2ae71d5730b6 None admin] PATH_INFO: `/v3/users/8fa31761548948049aa25bd5881382c4` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:33.068067 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9eb04e8-4abf-4adc-a2d1-2ae71d5730b6 None admin] RBAC: Authorizing `identity:get_user(user_id=8fa31761548948049aa25bd5881382c4)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:33.070626 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9eb04e8-4abf-4adc-a2d1-2ae71d5730b6 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:33.073025 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 62/128] 10.222.0.66 () {56 vars in 1177 bytes} [Mon Jul 18 02:14:32 2022] GET /identity/v3/users/8fa31761548948049aa25bd5881382c4 => generated 275 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:14:33.073985 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eb04e8-4abf-4adc-a2d1-2ae71d5730b6 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:33.090172 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9cb7d4-8c66-494c-8dc9-a58da1d109f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:33.090835 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9cb7d4-8c66-494c-8dc9-a58da1d109f4 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:33.091125 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9cb7d4-8c66-494c-8dc9-a58da1d109f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:33.091399 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9cb7d4-8c66-494c-8dc9-a58da1d109f4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:33.162823 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9cb7d4-8c66-494c-8dc9-a58da1d109f4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:33.164349 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9cb7d4-8c66-494c-8dc9-a58da1d109f4 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:33.206476 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eb04e8-4abf-4adc-a2d1-2ae71d5730b6 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:33.230471 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:33.242198 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8394078-21e1-4145-a124-fee9aee340a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:33.243098 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8394078-21e1-4145-a124-fee9aee340a5 None admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:33.243385 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8394078-21e1-4145-a124-fee9aee340a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:33.253932 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8394078-21e1-4145-a124-fee9aee340a5 None admin] PATH_INFO: `/v3/projects/a5370165a2354ba6aec1bb43138f4838/users/8fa31761548948049aa25bd5881382c4/roles/a5b895323e7b4f0db2491abe9ad57ee9` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:33.253932 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8394078-21e1-4145-a124-fee9aee340a5 None admin] RBAC: Authorizing `identity:create_grant(project_id=a5370165a2354ba6aec1bb43138f4838, user_id=8fa31761548948049aa25bd5881382c4, role_id=a5b895323e7b4f0db2491abe9ad57ee9)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:33.253932 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8394078-21e1-4145-a124-fee9aee340a5 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:33.266480 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:33.289129 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 67/129] 10.222.0.66 () {58 vars in 1282 bytes} [Mon Jul 18 02:14:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 02:14:33.291275 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9cb7d4-8c66-494c-8dc9-a58da1d109f4 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:33.367066 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 63/130] 10.222.0.66 () {58 vars in 1439 bytes} [Mon Jul 18 02:14:33 2022] PUT /identity/v3/projects/a5370165a2354ba6aec1bb43138f4838/users/8fa31761548948049aa25bd5881382c4/roles/a5b895323e7b4f0db2491abe9ad57ee9 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:14:33.368328 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8394078-21e1-4145-a124-fee9aee340a5 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:33.411921 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9cb7d4-8c66-494c-8dc9-a58da1d109f4 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:33.457015 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b429cf8-9f95-4dd2-847c-be9026619877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:33.458294 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b429cf8-9f95-4dd2-847c-be9026619877 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:33.458614 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b429cf8-9f95-4dd2-847c-be9026619877 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:33.460006 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b429cf8-9f95-4dd2-847c-be9026619877 None admin] PATH_INFO: `/v3/users/62dd67591e0147fe97b248079359a4de` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:33.461437 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b429cf8-9f95-4dd2-847c-be9026619877 None admin] RBAC: Authorizing `identity:get_user(user_id=62dd67591e0147fe97b248079359a4de)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:33.463029 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b429cf8-9f95-4dd2-847c-be9026619877 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:33.465207 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 68/131] 10.222.0.66 () {56 vars in 1177 bytes} [Mon Jul 18 02:14:33 2022] GET /identity/v3/users/62dd67591e0147fe97b248079359a4de => generated 247 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:14:33.469141 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b429cf8-9f95-4dd2-847c-be9026619877 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:33.496437 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8394078-21e1-4145-a124-fee9aee340a5 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:33.538176 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a31783b-0a93-43af-b7d9-e77d5a0f9863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:33.539887 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a31783b-0a93-43af-b7d9-e77d5a0f9863 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:33.540308 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a31783b-0a93-43af-b7d9-e77d5a0f9863 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:33.540783 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:33.540897 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a31783b-0a93-43af-b7d9-e77d5a0f9863 None admin] PATH_INFO: `/v3/projects/9b769253e8b5410b9c5fabbea39b7078` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:33.543200 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a31783b-0a93-43af-b7d9-e77d5a0f9863 None admin] RBAC: Authorizing `identity:get_project(project_id=9b769253e8b5410b9c5fabbea39b7078)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:33.545537 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a31783b-0a93-43af-b7d9-e77d5a0f9863 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:33.547481 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 64/132] 10.222.0.66 () {56 vars in 1186 bytes} [Mon Jul 18 02:14:33 2022] GET /identity/v3/projects/9b769253e8b5410b9c5fabbea39b7078 => generated 312 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:14:33.548946 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8749a5b-feb3-407d-8daa-16573cfd4fd9 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:33.549771 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8749a5b-feb3-407d-8daa-16573cfd4fd9 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:33.551075 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8749a5b-feb3-407d-8daa-16573cfd4fd9 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:33.552475 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 65/133] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:14:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:14:33.553216 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8749a5b-feb3-407d-8daa-16573cfd4fd9 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:33.616044 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b429cf8-9f95-4dd2-847c-be9026619877 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:33.623634 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:33.651594 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c66e555-6f7b-4cdc-9570-ac2ca76def81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:33.652499 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c66e555-6f7b-4cdc-9570-ac2ca76def81 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:33.652790 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c66e555-6f7b-4cdc-9570-ac2ca76def81 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:33.653075 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c66e555-6f7b-4cdc-9570-ac2ca76def81 None admin] PATH_INFO: `/v3/projects/a5370165a2354ba6aec1bb43138f4838` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:33.654323 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c66e555-6f7b-4cdc-9570-ac2ca76def81 None admin] RBAC: Authorizing `identity:get_project(project_id=a5370165a2354ba6aec1bb43138f4838)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:33.655818 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c66e555-6f7b-4cdc-9570-ac2ca76def81 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:33.657551 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 69/134] 10.222.0.66 () {56 vars in 1186 bytes} [Mon Jul 18 02:14:33 2022] GET /identity/v3/projects/a5370165a2354ba6aec1bb43138f4838 => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:14:33.658760 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4385183b-2a2b-437f-8747-bee85e710390 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:33.659099 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4385183b-2a2b-437f-8747-bee85e710390 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:33.662531 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4385183b-2a2b-437f-8747-bee85e710390 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:33.692708 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8749a5b-feb3-407d-8daa-16573cfd4fd9 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:33.693736 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-4385183b-2a2b-437f-8747-bee85e710390 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:14:33.695018 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-4385183b-2a2b-437f-8747-bee85e710390 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:14:33.732019 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2efec07-2ad6-48f9-bb13-0cb5a6a6721a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:33.733295 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2efec07-2ad6-48f9-bb13-0cb5a6a6721a None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:33.733757 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2efec07-2ad6-48f9-bb13-0cb5a6a6721a None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:33.734295 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2efec07-2ad6-48f9-bb13-0cb5a6a6721a None admin] PATH_INFO: `/v3/projects/a5370165a2354ba6aec1bb43138f4838` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:33.735608 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2efec07-2ad6-48f9-bb13-0cb5a6a6721a None admin] RBAC: Authorizing `identity:get_project(project_id=a5370165a2354ba6aec1bb43138f4838)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:33.737778 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2efec07-2ad6-48f9-bb13-0cb5a6a6721a None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:33.739670 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 66/135] 10.222.0.66 () {56 vars in 1186 bytes} [Mon Jul 18 02:14:33 2022] GET /identity/v3/projects/a5370165a2354ba6aec1bb43138f4838 => generated 298 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:14:33.740607 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2efec07-2ad6-48f9-bb13-0cb5a6a6721a None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:33.785555 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 70/136] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:14:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 02:14:33.786672 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4385183b-2a2b-437f-8747-bee85e710390 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:33.849816 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:33.874634 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2efec07-2ad6-48f9-bb13-0cb5a6a6721a None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:33.896995 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:33.917742 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4385183b-2a2b-437f-8747-bee85e710390 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:33.922934 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384d0d10-f8a1-467d-97fd-08d583727a2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:33.924283 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384d0d10-f8a1-467d-97fd-08d583727a2b None admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:33.925862 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384d0d10-f8a1-467d-97fd-08d583727a2b None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:33.926439 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384d0d10-f8a1-467d-97fd-08d583727a2b None admin] PATH_INFO: `/v3/projects/9b769253e8b5410b9c5fabbea39b7078/users/8fa31761548948049aa25bd5881382c4/roles/65feabf3b27944a8b01a86c8ac608457` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:33.929926 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384d0d10-f8a1-467d-97fd-08d583727a2b None admin] RBAC: Authorizing `identity:create_grant(project_id=9b769253e8b5410b9c5fabbea39b7078, user_id=8fa31761548948049aa25bd5881382c4, role_id=65feabf3b27944a8b01a86c8ac608457)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:33.932841 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384d0d10-f8a1-467d-97fd-08d583727a2b None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:33.956687 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47911ed4-18a2-4a57-9ad5-1755057164b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:33.957350 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47911ed4-18a2-4a57-9ad5-1755057164b8 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:33.957631 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47911ed4-18a2-4a57-9ad5-1755057164b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:33.957933 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47911ed4-18a2-4a57-9ad5-1755057164b8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:33.958802 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47911ed4-18a2-4a57-9ad5-1755057164b8 None admin] RBAC: Adding query filter params (role.id=4045191c71164a3e9a009f23ab3b1406, scope.project.id=a5370165a2354ba6aec1bb43138f4838, user.id=20fb40a9faec4e74bcb9275b58037782) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:14:33.959110 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47911ed4-18a2-4a57-9ad5-1755057164b8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:33.960552 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47911ed4-18a2-4a57-9ad5-1755057164b8 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:33.966277 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 71/137] 10.222.0.66 () {56 vars in 1374 bytes} [Mon Jul 18 02:14:33 2022] GET /identity/v3/role_assignments?user.id=20fb40a9faec4e74bcb9275b58037782&scope.project.id=a5370165a2354ba6aec1bb43138f4838&role.id=4045191c71164a3e9a009f23ab3b1406 => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:14:33.967028 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47911ed4-18a2-4a57-9ad5-1755057164b8 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:34.037072 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 67/138] 10.222.0.66 () {58 vars in 1439 bytes} [Mon Jul 18 02:14:33 2022] PUT /identity/v3/projects/9b769253e8b5410b9c5fabbea39b7078/users/8fa31761548948049aa25bd5881382c4/roles/65feabf3b27944a8b01a86c8ac608457 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:14:34.038587 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384d0d10-f8a1-467d-97fd-08d583727a2b None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:34.094215 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47911ed4-18a2-4a57-9ad5-1755057164b8 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:34.099956 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:34.136411 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484b121b-d476-41b2-807a-445146ffb162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:34.137603 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484b121b-d476-41b2-807a-445146ffb162 None admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:34.138073 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484b121b-d476-41b2-807a-445146ffb162 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:34.138530 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484b121b-d476-41b2-807a-445146ffb162 None admin] PATH_INFO: `/v3/projects/a5370165a2354ba6aec1bb43138f4838/users/62dd67591e0147fe97b248079359a4de/roles/d4aa3324f0c84887b41d0dc4acf7f30e` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:34.144689 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484b121b-d476-41b2-807a-445146ffb162 None admin] RBAC: Authorizing `identity:create_grant(project_id=a5370165a2354ba6aec1bb43138f4838, user_id=62dd67591e0147fe97b248079359a4de, role_id=d4aa3324f0c84887b41d0dc4acf7f30e)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:34.146213 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484b121b-d476-41b2-807a-445146ffb162 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:34.197609 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384d0d10-f8a1-467d-97fd-08d583727a2b None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:34.237168 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc7fc2d-f1f3-4e93-aeac-20f499371642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:34.238176 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc7fc2d-f1f3-4e93-aeac-20f499371642 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:34.238568 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc7fc2d-f1f3-4e93-aeac-20f499371642 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:34.239083 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc7fc2d-f1f3-4e93-aeac-20f499371642 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:34.240943 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc7fc2d-f1f3-4e93-aeac-20f499371642 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:34.242766 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc7fc2d-f1f3-4e93-aeac-20f499371642 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:34.244848 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 68/139] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 18 02:14:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:14:34.250349 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc7fc2d-f1f3-4e93-aeac-20f499371642 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:34.265982 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 72/140] 10.222.0.66 () {58 vars in 1439 bytes} [Mon Jul 18 02:14:33 2022] PUT /identity/v3/projects/a5370165a2354ba6aec1bb43138f4838/users/62dd67591e0147fe97b248079359a4de/roles/d4aa3324f0c84887b41d0dc4acf7f30e => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:14:34.305903 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:34.372193 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc7fc2d-f1f3-4e93-aeac-20f499371642 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:34.413114 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42ac117-4e36-4033-a54c-4606d1907b9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:34.419218 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42ac117-4e36-4033-a54c-4606d1907b9d None admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:34.419218 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42ac117-4e36-4033-a54c-4606d1907b9d None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:34.419218 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42ac117-4e36-4033-a54c-4606d1907b9d None admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:34.419218 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42ac117-4e36-4033-a54c-4606d1907b9d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:34.419218 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42ac117-4e36-4033-a54c-4606d1907b9d None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:34.545239 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 69/141] 10.222.0.66 () {60 vars in 1132 bytes} [Mon Jul 18 02:14:34 2022] POST /identity/v3/users => generated 283 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:14:34.970549 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddd57ae-ea83-41ca-842d-6dedf3318467 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:34.971091 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddd57ae-ea83-41ca-842d-6dedf3318467 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:34.971091 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddd57ae-ea83-41ca-842d-6dedf3318467 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:34.973128 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 73/142] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:14:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:14:34.978519 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96650243-ded5-4e5b-8a24-8555993f0fa3 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:34.978941 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96650243-ded5-4e5b-8a24-8555993f0fa3 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:34.979283 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96650243-ded5-4e5b-8a24-8555993f0fa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:35.001184 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-96650243-ded5-4e5b-8a24-8555993f0fa3 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:14:35.002480 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-96650243-ded5-4e5b-8a24-8555993f0fa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:14:35.079851 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 70/143] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:14:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 02:14:35.123510 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddd57ae-ea83-41ca-842d-6dedf3318467 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:35.244419 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddd57ae-ea83-41ca-842d-6dedf3318467 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:35.282231 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214ea31e-25dc-47ff-993f-0fb284c037b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:35.283412 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214ea31e-25dc-47ff-993f-0fb284c037b5 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:35.284000 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214ea31e-25dc-47ff-993f-0fb284c037b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:35.284511 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214ea31e-25dc-47ff-993f-0fb284c037b5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:35.301844 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214ea31e-25dc-47ff-993f-0fb284c037b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 02:14:35.309724 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-214ea31e-25dc-47ff-993f-0fb284c037b5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 02:14:35.309724 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:14:35.309724 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:14:35.309724 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:14:35.309724 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:14:35.309724 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:14:35.309724 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:14:35.309724 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:14:35.309724 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:14:35.309724 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:14:35.309724 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:14:35.309724 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:14:35.309724 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 02:14:35.309724 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 02:14:35.309724 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 02:14:35.309724 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 02:14:35.310602 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 02:14:35.310602 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 02:14:35.310602 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 02:14:35.310602 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 02:14:35.310602 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:14:35.310602 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:14:35.310602 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:14:35.310602 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:14:35.310602 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:14:35.310602 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:14:35.310602 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:14:35.310602 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:14:35.310602 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:14:35.310602 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:14:35.310602 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:14:35.310602 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:14:35.311365 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:14:35.311365 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:14:35.311365 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:14:35.311365 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:14:35.311365 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 02:14:35.311365 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 02:14:35.311365 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 02:14:35.311365 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 02:14:35.311365 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 02:14:35.311365 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 02:14:35.311365 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 02:14:35.311365 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:14:35.313683 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 74/144] 10.222.0.66 () {56 vars in 1099 bytes} [Mon Jul 18 02:14:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:14:35.322784 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96650243-ded5-4e5b-8a24-8555993f0fa3 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:35.370520 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:35.458695 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96650243-ded5-4e5b-8a24-8555993f0fa3 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:35.500281 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458639b5-18c6-4eab-b828-82c098b5ec7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:35.501059 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458639b5-18c6-4eab-b828-82c098b5ec7e None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:35.501348 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458639b5-18c6-4eab-b828-82c098b5ec7e None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:35.501630 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458639b5-18c6-4eab-b828-82c098b5ec7e None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:35.502432 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458639b5-18c6-4eab-b828-82c098b5ec7e None admin] RBAC: Adding query filter params (name=member) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:14:35.503331 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458639b5-18c6-4eab-b828-82c098b5ec7e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:35.504794 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458639b5-18c6-4eab-b828-82c098b5ec7e None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:35.515035 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 71/145] 10.222.0.66 () {56 vars in 1101 bytes} [Mon Jul 18 02:14:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:14:35.523229 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214ea31e-25dc-47ff-993f-0fb284c037b5 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:35.568738 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:35.669178 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214ea31e-25dc-47ff-993f-0fb284c037b5 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:35.727335 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3107f078-a793-460d-9c05-67835754fd48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:35.728228 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3107f078-a793-460d-9c05-67835754fd48 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:35.728526 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3107f078-a793-460d-9c05-67835754fd48 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:35.728804 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3107f078-a793-460d-9c05-67835754fd48 None admin] PATH_INFO: `/v3/users/8fa31761548948049aa25bd5881382c4` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:35.734834 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3107f078-a793-460d-9c05-67835754fd48 None admin] RBAC: Authorizing `identity:get_user(user_id=8fa31761548948049aa25bd5881382c4)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:35.736329 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3107f078-a793-460d-9c05-67835754fd48 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:35.738248 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 75/146] 10.222.0.66 () {56 vars in 1177 bytes} [Mon Jul 18 02:14:35 2022] GET /identity/v3/users/8fa31761548948049aa25bd5881382c4 => generated 275 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:14:35.743833 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458639b5-18c6-4eab-b828-82c098b5ec7e None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:35.876145 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:35.890572 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458639b5-18c6-4eab-b828-82c098b5ec7e None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:35.939389 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7201bf-4654-40c7-91cc-c9e60dfcac2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:35.940107 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7201bf-4654-40c7-91cc-c9e60dfcac2c None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:35.941029 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7201bf-4654-40c7-91cc-c9e60dfcac2c None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:35.941315 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7201bf-4654-40c7-91cc-c9e60dfcac2c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:36.038524 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7201bf-4654-40c7-91cc-c9e60dfcac2c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:36.040021 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7201bf-4654-40c7-91cc-c9e60dfcac2c None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:36.161984 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 72/147] 10.222.0.66 () {58 vars in 1282 bytes} [Mon Jul 18 02:14:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 02:14:36.162668 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:36.171158 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3107f078-a793-460d-9c05-67835754fd48 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:36.228429 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582cff6d-6990-405d-bd8f-d910c80eb3ed None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:36.229070 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582cff6d-6990-405d-bd8f-d910c80eb3ed None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:36.229554 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582cff6d-6990-405d-bd8f-d910c80eb3ed None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:36.231238 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 73/148] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:14:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:14:36.236857 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0a1f99-36d5-47f7-a8ae-5ee4e0fef158 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:36.237955 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0a1f99-36d5-47f7-a8ae-5ee4e0fef158 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:36.238536 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0a1f99-36d5-47f7-a8ae-5ee4e0fef158 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:36.259018 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-cc0a1f99-36d5-47f7-a8ae-5ee4e0fef158 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:14:36.260554 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-cc0a1f99-36d5-47f7-a8ae-5ee4e0fef158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:14:36.350372 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:36.360314 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3107f078-a793-460d-9c05-67835754fd48 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:36.374046 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 74/149] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:14:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 02:14:36.405909 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ad37e3-944e-4472-a3a0-7f85ac6d8435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:36.407333 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ad37e3-944e-4472-a3a0-7f85ac6d8435 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:36.407792 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ad37e3-944e-4472-a3a0-7f85ac6d8435 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:36.408228 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ad37e3-944e-4472-a3a0-7f85ac6d8435 None admin] PATH_INFO: `/v3/projects/a5370165a2354ba6aec1bb43138f4838` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:36.417215 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ad37e3-944e-4472-a3a0-7f85ac6d8435 None admin] RBAC: Authorizing `identity:get_project(project_id=a5370165a2354ba6aec1bb43138f4838)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:36.417215 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ad37e3-944e-4472-a3a0-7f85ac6d8435 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:36.418657 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 76/150] 10.222.0.66 () {56 vars in 1186 bytes} [Mon Jul 18 02:14:36 2022] GET /identity/v3/projects/a5370165a2354ba6aec1bb43138f4838 => generated 298 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:14:36.425498 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0a1f99-36d5-47f7-a8ae-5ee4e0fef158 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:36.461185 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ad37e3-944e-4472-a3a0-7f85ac6d8435 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:36.653484 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0a1f99-36d5-47f7-a8ae-5ee4e0fef158 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:36.670138 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ad37e3-944e-4472-a3a0-7f85ac6d8435 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:36.703868 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:36.726524 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93894f8e-764b-49a4-9e07-b617defeb156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:36.728573 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93894f8e-764b-49a4-9e07-b617defeb156 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:36.730580 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93894f8e-764b-49a4-9e07-b617defeb156 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:36.731168 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93894f8e-764b-49a4-9e07-b617defeb156 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:36.733109 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93894f8e-764b-49a4-9e07-b617defeb156 None admin] RBAC: Adding query filter params (role.id=65feabf3b27944a8b01a86c8ac608457, scope.project.id=a5370165a2354ba6aec1bb43138f4838, user.id=8fa31761548948049aa25bd5881382c4) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:14:36.734305 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93894f8e-764b-49a4-9e07-b617defeb156 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:36.736200 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93894f8e-764b-49a4-9e07-b617defeb156 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:36.745371 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3101723-b4e7-4a2a-8814-73ab7893d2b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:36.746522 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3101723-b4e7-4a2a-8814-73ab7893d2b1 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:36.747010 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3101723-b4e7-4a2a-8814-73ab7893d2b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:36.747468 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3101723-b4e7-4a2a-8814-73ab7893d2b1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:36.752168 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 75/151] 10.222.0.66 () {56 vars in 1374 bytes} [Mon Jul 18 02:14:36 2022] GET /identity/v3/role_assignments?user.id=8fa31761548948049aa25bd5881382c4&scope.project.id=a5370165a2354ba6aec1bb43138f4838&role.id=65feabf3b27944a8b01a86c8ac608457 => generated 608 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:14:36.763439 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3101723-b4e7-4a2a-8814-73ab7893d2b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 02:14:36.772592 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-f3101723-b4e7-4a2a-8814-73ab7893d2b1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 02:14:36.772592 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:14:36.772592 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:14:36.772592 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:14:36.772592 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:14:36.772592 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:14:36.772592 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:14:36.772592 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:14:36.772592 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:14:36.772592 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:14:36.772592 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:14:36.772592 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:14:36.772592 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 02:14:36.772592 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 02:14:36.772592 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 02:14:36.772592 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 02:14:36.773548 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 02:14:36.773548 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 02:14:36.773548 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 02:14:36.773548 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 02:14:36.773548 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:14:36.773548 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:14:36.773548 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:14:36.773548 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:14:36.773548 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:14:36.773548 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:14:36.773548 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:14:36.773548 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:14:36.773548 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:14:36.773548 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:14:36.773548 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:14:36.773548 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:14:36.774394 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:14:36.774394 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:14:36.774394 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:14:36.774394 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:14:36.774394 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 02:14:36.774394 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 02:14:36.774394 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 02:14:36.774394 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 02:14:36.774394 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 02:14:36.774394 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 02:14:36.774394 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 02:14:36.774394 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:14:36.776983 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 77/152] 10.222.0.66 () {56 vars in 1114 bytes} [Mon Jul 18 02:14:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 02:14:36.783248 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93894f8e-764b-49a4-9e07-b617defeb156 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:36.920082 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93894f8e-764b-49a4-9e07-b617defeb156 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:36.966370 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b781ed5-cc93-46db-9f35-34a3dabcef73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:36.967506 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b781ed5-cc93-46db-9f35-34a3dabcef73 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:36.968004 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b781ed5-cc93-46db-9f35-34a3dabcef73 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:36.968485 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b781ed5-cc93-46db-9f35-34a3dabcef73 None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:36.969453 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b781ed5-cc93-46db-9f35-34a3dabcef73 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:14:36.969977 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b781ed5-cc93-46db-9f35-34a3dabcef73 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:36.972340 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b781ed5-cc93-46db-9f35-34a3dabcef73 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:36.982458 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 76/153] 10.222.0.66 () {56 vars in 1111 bytes} [Mon Jul 18 02:14:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:14:36.991254 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3101723-b4e7-4a2a-8814-73ab7893d2b1 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:37.024790 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:37.130066 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3101723-b4e7-4a2a-8814-73ab7893d2b1 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:37.175932 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c55aae2-6de8-4102-a613-5aa66f9a58ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:37.176841 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c55aae2-6de8-4102-a613-5aa66f9a58ed None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:37.177211 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c55aae2-6de8-4102-a613-5aa66f9a58ed None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:37.177475 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c55aae2-6de8-4102-a613-5aa66f9a58ed None admin] PATH_INFO: `/v3/users/8fa31761548948049aa25bd5881382c4` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:37.182816 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c55aae2-6de8-4102-a613-5aa66f9a58ed None admin] RBAC: Authorizing `identity:get_user(user_id=8fa31761548948049aa25bd5881382c4)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:37.184303 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c55aae2-6de8-4102-a613-5aa66f9a58ed None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:37.186002 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 78/154] 10.222.0.66 () {56 vars in 1177 bytes} [Mon Jul 18 02:14:36 2022] GET /identity/v3/users/8fa31761548948049aa25bd5881382c4 => generated 275 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:14:37.191774 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b781ed5-cc93-46db-9f35-34a3dabcef73 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:37.254684 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:37.328665 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b781ed5-cc93-46db-9f35-34a3dabcef73 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:37.387921 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1142fe49-0425-421d-9416-b69e78698a88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:37.388613 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1142fe49-0425-421d-9416-b69e78698a88 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:37.388899 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1142fe49-0425-421d-9416-b69e78698a88 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:37.389180 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1142fe49-0425-421d-9416-b69e78698a88 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:37.482934 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1142fe49-0425-421d-9416-b69e78698a88 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:37.484977 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1142fe49-0425-421d-9416-b69e78698a88 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:37.539844 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ef9dc0-5086-4995-9646-0aaf40d7f8b4 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:37.540137 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ef9dc0-5086-4995-9646-0aaf40d7f8b4 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:37.540373 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ef9dc0-5086-4995-9646-0aaf40d7f8b4 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:37.541786 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 79/155] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:14:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:14:37.548156 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97914413-4cf6-4df1-9761-016dd26a3e76 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:37.548498 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97914413-4cf6-4df1-9761-016dd26a3e76 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:37.548780 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97914413-4cf6-4df1-9761-016dd26a3e76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:37.584845 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:37.587417 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-97914413-4cf6-4df1-9761-016dd26a3e76 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:14:37.588618 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-97914413-4cf6-4df1-9761-016dd26a3e76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:14:37.744760 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 77/156] 10.222.0.66 () {58 vars in 1282 bytes} [Mon Jul 18 02:14:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 553 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 02:14:37.751840 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1142fe49-0425-421d-9416-b69e78698a88 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:37.871913 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 80/157] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:14:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 02:14:37.873331 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820aa08f-2888-4a03-b645-b7b59e963f44 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:37.873626 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820aa08f-2888-4a03-b645-b7b59e963f44 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:37.873930 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820aa08f-2888-4a03-b645-b7b59e963f44 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:37.875266 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 81/158] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:14:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:14:37.887455 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3551e098-2135-4aef-b3f7-d135b08902f2 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:37.887749 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3551e098-2135-4aef-b3f7-d135b08902f2 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:37.888035 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3551e098-2135-4aef-b3f7-d135b08902f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:37.931130 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-3551e098-2135-4aef-b3f7-d135b08902f2 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:14:37.932345 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-3551e098-2135-4aef-b3f7-d135b08902f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:14:37.979689 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:38.057070 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1142fe49-0425-421d-9416-b69e78698a88 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:38.128192 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 82/159] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:14:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 02:14:38.129248 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3551e098-2135-4aef-b3f7-d135b08902f2 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:38.170832 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e32d9f3-2cf3-4fbc-8eea-78a772dec873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:38.172097 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e32d9f3-2cf3-4fbc-8eea-78a772dec873 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:38.172706 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e32d9f3-2cf3-4fbc-8eea-78a772dec873 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:38.173174 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e32d9f3-2cf3-4fbc-8eea-78a772dec873 None admin] PATH_INFO: `/v3/projects/a5370165a2354ba6aec1bb43138f4838` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:38.175772 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e32d9f3-2cf3-4fbc-8eea-78a772dec873 None admin] RBAC: Authorizing `identity:get_project(project_id=a5370165a2354ba6aec1bb43138f4838)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:38.177545 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e32d9f3-2cf3-4fbc-8eea-78a772dec873 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:38.183264 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 78/160] 10.222.0.66 () {56 vars in 1186 bytes} [Mon Jul 18 02:14:37 2022] GET /identity/v3/projects/a5370165a2354ba6aec1bb43138f4838 => generated 298 bytes in 432 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:14:38.184302 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e32d9f3-2cf3-4fbc-8eea-78a772dec873 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:38.213546 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:38.366271 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e32d9f3-2cf3-4fbc-8eea-78a772dec873 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:38.372825 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3551e098-2135-4aef-b3f7-d135b08902f2 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:38.435081 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f4c27a-5b2a-448d-9908-07702f172aad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:38.436419 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f4c27a-5b2a-448d-9908-07702f172aad None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:38.436904 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f4c27a-5b2a-448d-9908-07702f172aad None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:38.437416 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f4c27a-5b2a-448d-9908-07702f172aad None admin] PATH_INFO: `/v3/roles/member` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:38.449096 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daecd6ab-04e9-4f00-a91f-129ff8fd7e83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:38.449903 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daecd6ab-04e9-4f00-a91f-129ff8fd7e83 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:38.450212 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daecd6ab-04e9-4f00-a91f-129ff8fd7e83 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:38.450489 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daecd6ab-04e9-4f00-a91f-129ff8fd7e83 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:38.459403 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f4c27a-5b2a-448d-9908-07702f172aad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 02:14:38.468060 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-38f4c27a-5b2a-448d-9908-07702f172aad None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 02:14:38.468060 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:14:38.468060 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:14:38.468060 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:14:38.468060 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:14:38.468060 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:14:38.468060 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:14:38.468060 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:14:38.468060 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:14:38.468060 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:14:38.468060 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:14:38.468060 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:14:38.468060 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 02:14:38.468060 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 02:14:38.468060 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 02:14:38.468060 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 02:14:38.469035 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 02:14:38.469035 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 02:14:38.469035 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 02:14:38.469035 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 02:14:38.469035 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:14:38.469035 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:14:38.469035 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:14:38.469035 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:14:38.469035 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:14:38.469035 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:14:38.469035 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:14:38.469035 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:14:38.469035 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:14:38.469035 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:14:38.469035 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:14:38.469035 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:14:38.469873 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:14:38.469873 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:14:38.469873 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:14:38.469873 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:14:38.469873 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 02:14:38.469873 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 02:14:38.469873 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 02:14:38.469873 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 02:14:38.469873 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 02:14:38.469873 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 02:14:38.469873 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 02:14:38.469873 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:14:38.473118 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 79/161] 10.222.0.66 () {56 vars in 1099 bytes} [Mon Jul 18 02:14:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:14:38.474267 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f4c27a-5b2a-448d-9908-07702f172aad None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:38.480387 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daecd6ab-04e9-4f00-a91f-129ff8fd7e83 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 02:14:38.489810 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-daecd6ab-04e9-4f00-a91f-129ff8fd7e83 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 02:14:38.489810 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:14:38.489810 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:14:38.489810 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:14:38.489810 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:14:38.489810 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:14:38.489810 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:14:38.489810 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:14:38.489810 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:14:38.489810 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:14:38.489810 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:14:38.489810 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:14:38.489810 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 02:14:38.489810 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 02:14:38.489810 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 02:14:38.489810 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 02:14:38.490769 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 02:14:38.490769 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 02:14:38.490769 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 02:14:38.490769 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 02:14:38.490769 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:14:38.490769 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:14:38.490769 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:14:38.490769 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:14:38.490769 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:14:38.490769 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:14:38.490769 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:14:38.490769 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:14:38.490769 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:14:38.490769 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:14:38.490769 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:14:38.490769 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:14:38.491552 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:14:38.491552 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:14:38.491552 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:14:38.491552 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:14:38.491552 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 02:14:38.491552 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 02:14:38.491552 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 02:14:38.491552 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 02:14:38.491552 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 02:14:38.491552 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 02:14:38.491552 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 02:14:38.491552 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:14:38.491552 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 83/162] 10.222.0.66 () {56 vars in 1099 bytes} [Mon Jul 18 02:14:38 2022] GET /identity/v3/roles/reader => generated 84 bytes in 363 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:14:38.492228 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daecd6ab-04e9-4f00-a91f-129ff8fd7e83 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:38.528101 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:38.636042 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:38.693578 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f4c27a-5b2a-448d-9908-07702f172aad None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:38.709669 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daecd6ab-04e9-4f00-a91f-129ff8fd7e83 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:38.743634 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc92544-0dbf-4757-a592-ac66f862b510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:38.744306 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc92544-0dbf-4757-a592-ac66f862b510 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:38.744586 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc92544-0dbf-4757-a592-ac66f862b510 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:38.744864 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc92544-0dbf-4757-a592-ac66f862b510 None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:38.745613 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc92544-0dbf-4757-a592-ac66f862b510 None admin] RBAC: Adding query filter params (name=member) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:14:38.745915 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc92544-0dbf-4757-a592-ac66f862b510 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:38.747343 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc92544-0dbf-4757-a592-ac66f862b510 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:38.756905 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 84/163] 10.222.0.66 () {56 vars in 1101 bytes} [Mon Jul 18 02:14:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:14:38.757723 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc92544-0dbf-4757-a592-ac66f862b510 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:38.769756 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c278e1bd-5bd8-485e-acbb-637cd00169d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:38.770720 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c278e1bd-5bd8-485e-acbb-637cd00169d5 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:38.771000 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c278e1bd-5bd8-485e-acbb-637cd00169d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:38.771269 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c278e1bd-5bd8-485e-acbb-637cd00169d5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:38.772117 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c278e1bd-5bd8-485e-acbb-637cd00169d5 None admin] RBAC: Adding query filter params (role.id=a5b895323e7b4f0db2491abe9ad57ee9, scope.project.id=a5370165a2354ba6aec1bb43138f4838, user.id=8fa31761548948049aa25bd5881382c4) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:14:38.772427 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c278e1bd-5bd8-485e-acbb-637cd00169d5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:38.774117 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c278e1bd-5bd8-485e-acbb-637cd00169d5 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:38.780335 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 80/164] 10.222.0.66 () {56 vars in 1374 bytes} [Mon Jul 18 02:14:38 2022] GET /identity/v3/role_assignments?user.id=8fa31761548948049aa25bd5881382c4&scope.project.id=a5370165a2354ba6aec1bb43138f4838&role.id=a5b895323e7b4f0db2491abe9ad57ee9 => generated 608 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:14:38.781649 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def09670-1d0c-49cd-a301-6bccab2bd5cb None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:38.781942 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def09670-1d0c-49cd-a301-6bccab2bd5cb None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:38.782245 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def09670-1d0c-49cd-a301-6bccab2bd5cb None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:38.784263 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 81/165] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:14:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:14:38.790118 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def09670-1d0c-49cd-a301-6bccab2bd5cb None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:38.922500 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:38.960294 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc92544-0dbf-4757-a592-ac66f862b510 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:39.024518 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b99255-1ea8-4381-a1ec-ff728aa7ce90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:39.025304 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b99255-1ea8-4381-a1ec-ff728aa7ce90 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:39.025590 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b99255-1ea8-4381-a1ec-ff728aa7ce90 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:39.025864 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b99255-1ea8-4381-a1ec-ff728aa7ce90 None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:39.026643 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b99255-1ea8-4381-a1ec-ff728aa7ce90 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:14:39.026947 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b99255-1ea8-4381-a1ec-ff728aa7ce90 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:39.028322 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b99255-1ea8-4381-a1ec-ff728aa7ce90 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:39.037222 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 85/166] 10.222.0.66 () {56 vars in 1101 bytes} [Mon Jul 18 02:14:38 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:14:39.038561 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbaaffe-058d-4274-a26d-219a3892f6d7 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:39.038851 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbaaffe-058d-4274-a26d-219a3892f6d7 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:39.039248 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbaaffe-058d-4274-a26d-219a3892f6d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:39.057452 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-bbbaaffe-058d-4274-a26d-219a3892f6d7 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:14:39.058657 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-bbbaaffe-058d-4274-a26d-219a3892f6d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:14:39.075748 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def09670-1d0c-49cd-a301-6bccab2bd5cb None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:39.144890 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 86/167] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:14:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 02:14:39.145746 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbaaffe-058d-4274-a26d-219a3892f6d7 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:39.157467 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a22ec51-a08d-4608-8f4b-1292d7ca4779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:39.158392 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a22ec51-a08d-4608-8f4b-1292d7ca4779 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:39.159435 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a22ec51-a08d-4608-8f4b-1292d7ca4779 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:39.159814 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a22ec51-a08d-4608-8f4b-1292d7ca4779 None admin] PATH_INFO: `/v3/users/8fa31761548948049aa25bd5881382c4` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:39.163187 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a22ec51-a08d-4608-8f4b-1292d7ca4779 None admin] RBAC: Authorizing `identity:get_user(user_id=8fa31761548948049aa25bd5881382c4)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:39.164681 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a22ec51-a08d-4608-8f4b-1292d7ca4779 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:39.166393 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 82/168] 10.222.0.66 () {56 vars in 1177 bytes} [Mon Jul 18 02:14:38 2022] GET /identity/v3/users/8fa31761548948049aa25bd5881382c4 => generated 275 bytes in 382 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:14:39.172251 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a22ec51-a08d-4608-8f4b-1292d7ca4779 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:39.188634 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:39.238753 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:39.276808 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbaaffe-058d-4274-a26d-219a3892f6d7 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:39.296052 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a22ec51-a08d-4608-8f4b-1292d7ca4779 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:39.315946 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a044b8d8-5ad9-4b63-9746-91575ac44ede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:39.316924 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a044b8d8-5ad9-4b63-9746-91575ac44ede None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:39.317205 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a044b8d8-5ad9-4b63-9746-91575ac44ede None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:39.317475 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a044b8d8-5ad9-4b63-9746-91575ac44ede None admin] PATH_INFO: `/v3/users/20fb40a9faec4e74bcb9275b58037782` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:39.318712 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a044b8d8-5ad9-4b63-9746-91575ac44ede None admin] RBAC: Authorizing `identity:get_user(user_id=20fb40a9faec4e74bcb9275b58037782)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:39.320266 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a044b8d8-5ad9-4b63-9746-91575ac44ede None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:39.323090 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 87/169] 10.222.0.66 () {56 vars in 1177 bytes} [Mon Jul 18 02:14:39 2022] GET /identity/v3/users/20fb40a9faec4e74bcb9275b58037782 => generated 289 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:14:39.324311 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492cd6e7-dc52-41c2-ac2c-5db06d1f709a None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:39.324591 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492cd6e7-dc52-41c2-ac2c-5db06d1f709a None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:39.324863 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492cd6e7-dc52-41c2-ac2c-5db06d1f709a None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:39.326367 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 88/170] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:14:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:14:39.327054 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492cd6e7-dc52-41c2-ac2c-5db06d1f709a None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:39.338901 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb48d1f4-073e-43ce-bf6c-723873bbc55c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:39.339571 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb48d1f4-073e-43ce-bf6c-723873bbc55c None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:39.344476 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb48d1f4-073e-43ce-bf6c-723873bbc55c None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:39.344476 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb48d1f4-073e-43ce-bf6c-723873bbc55c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:39.417745 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb48d1f4-073e-43ce-bf6c-723873bbc55c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:39.419290 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb48d1f4-073e-43ce-bf6c-723873bbc55c None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:39.430035 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:39.441128 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492cd6e7-dc52-41c2-ac2c-5db06d1f709a None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:39.476236 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c282b94f-f059-4a41-b175-1ee84441ceba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:39.477262 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c282b94f-f059-4a41-b175-1ee84441ceba None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:39.477545 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c282b94f-f059-4a41-b175-1ee84441ceba None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:39.477817 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c282b94f-f059-4a41-b175-1ee84441ceba None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:39.479687 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:39.488752 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c282b94f-f059-4a41-b175-1ee84441ceba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 02:14:39.496377 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-c282b94f-f059-4a41-b175-1ee84441ceba None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 02:14:39.496377 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:14:39.496377 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:14:39.496377 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:14:39.496377 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:14:39.496377 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:14:39.496377 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:14:39.496377 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:14:39.496377 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:14:39.496377 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:14:39.496377 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:14:39.496377 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:14:39.496377 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 02:14:39.496377 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 02:14:39.496377 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 02:14:39.496377 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 02:14:39.497413 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 02:14:39.497413 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 02:14:39.497413 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 02:14:39.497413 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 02:14:39.497413 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:14:39.497413 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:14:39.497413 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:14:39.497413 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:14:39.497413 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:14:39.497413 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:14:39.497413 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:14:39.497413 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:14:39.497413 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:14:39.497413 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:14:39.497413 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:14:39.497413 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:14:39.498709 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:14:39.498709 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:14:39.498709 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:14:39.498709 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:14:39.498709 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 02:14:39.498709 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 02:14:39.498709 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 02:14:39.498709 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 02:14:39.498709 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 02:14:39.498709 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 02:14:39.498709 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 02:14:39.498709 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:14:39.498709 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 89/171] 10.222.0.66 () {56 vars in 1096 bytes} [Mon Jul 18 02:14:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:14:39.498709 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c282b94f-f059-4a41-b175-1ee84441ceba None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:39.524721 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 83/172] 10.222.0.66 () {58 vars in 1282 bytes} [Mon Jul 18 02:14:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 02:14:39.526642 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9de1ca-e563-4bbe-b788-0ea33fb5925f None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:39.526943 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9de1ca-e563-4bbe-b788-0ea33fb5925f None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:39.527224 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9de1ca-e563-4bbe-b788-0ea33fb5925f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:39.543834 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-de9de1ca-e563-4bbe-b788-0ea33fb5925f None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:14:39.545021 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-de9de1ca-e563-4bbe-b788-0ea33fb5925f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:14:39.601525 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c282b94f-f059-4a41-b175-1ee84441ceba None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:39.612690 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 84/173] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:14:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 02:14:39.615482 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac84e182-0c53-425f-b6f6-e634a7821fd7 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:39.615778 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac84e182-0c53-425f-b6f6-e634a7821fd7 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:39.616064 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac84e182-0c53-425f-b6f6-e634a7821fd7 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:39.617605 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 85/174] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:14:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:14:39.618798 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab5df0a-d164-4577-a2d7-9659c5fca5ff None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:39.619098 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab5df0a-d164-4577-a2d7-9659c5fca5ff None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:39.619388 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab5df0a-d164-4577-a2d7-9659c5fca5ff None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:39.620756 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 86/175] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:14:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:14:39.622873 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab5df0a-d164-4577-a2d7-9659c5fca5ff None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:39.640868 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a65f62-05ac-4dc7-9a24-27ab160d82af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:39.641724 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a65f62-05ac-4dc7-9a24-27ab160d82af None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:39.642001 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a65f62-05ac-4dc7-9a24-27ab160d82af None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:39.642308 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a65f62-05ac-4dc7-9a24-27ab160d82af None admin] PATH_INFO: `/v3/projects/a5370165a2354ba6aec1bb43138f4838` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:39.643987 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a65f62-05ac-4dc7-9a24-27ab160d82af None admin] RBAC: Authorizing `identity:get_project(project_id=a5370165a2354ba6aec1bb43138f4838)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:39.645443 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a65f62-05ac-4dc7-9a24-27ab160d82af None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:39.647188 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 90/176] 10.222.0.66 () {56 vars in 1186 bytes} [Mon Jul 18 02:14:39 2022] GET /identity/v3/projects/a5370165a2354ba6aec1bb43138f4838 => generated 298 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:14:39.650491 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a65f62-05ac-4dc7-9a24-27ab160d82af None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:39.666759 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:39.711941 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:39.727225 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab5df0a-d164-4577-a2d7-9659c5fca5ff None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:39.750341 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a65f62-05ac-4dc7-9a24-27ab160d82af None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:39.761236 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23842501-e2a4-4618-a3d5-15e30767b0aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:39.761900 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23842501-e2a4-4618-a3d5-15e30767b0aa None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:39.762225 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23842501-e2a4-4618-a3d5-15e30767b0aa None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:39.762505 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23842501-e2a4-4618-a3d5-15e30767b0aa None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:39.763253 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23842501-e2a4-4618-a3d5-15e30767b0aa None admin] RBAC: Adding query filter params (name=admin) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:14:39.763561 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23842501-e2a4-4618-a3d5-15e30767b0aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:39.764953 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23842501-e2a4-4618-a3d5-15e30767b0aa None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:39.777968 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 87/177] 10.222.0.66 () {56 vars in 1099 bytes} [Mon Jul 18 02:14:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:14:39.777968 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4241f95f-00f5-4f02-8115-3f6a10353037 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:39.777968 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4241f95f-00f5-4f02-8115-3f6a10353037 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:39.777968 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4241f95f-00f5-4f02-8115-3f6a10353037 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:39.783018 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270cc11d-937e-478e-bf14-b5b0b45436d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:39.784172 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270cc11d-937e-478e-bf14-b5b0b45436d7 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:39.784491 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270cc11d-937e-478e-bf14-b5b0b45436d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:39.784771 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270cc11d-937e-478e-bf14-b5b0b45436d7 None admin] PATH_INFO: `/v3/projects/9b769253e8b5410b9c5fabbea39b7078` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:39.786035 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270cc11d-937e-478e-bf14-b5b0b45436d7 None admin] RBAC: Authorizing `identity:get_project(project_id=9b769253e8b5410b9c5fabbea39b7078)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:39.787561 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270cc11d-937e-478e-bf14-b5b0b45436d7 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:39.789337 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 91/178] 10.222.0.66 () {56 vars in 1186 bytes} [Mon Jul 18 02:14:39 2022] GET /identity/v3/projects/9b769253e8b5410b9c5fabbea39b7078 => generated 312 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:14:39.790538 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdd429d-e26f-4200-8f3a-e348e9daac47 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:39.790826 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdd429d-e26f-4200-8f3a-e348e9daac47 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:39.791106 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdd429d-e26f-4200-8f3a-e348e9daac47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:39.795386 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-4241f95f-00f5-4f02-8115-3f6a10353037 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:14:39.796593 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-4241f95f-00f5-4f02-8115-3f6a10353037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:14:39.808125 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-6bdd429d-e26f-4200-8f3a-e348e9daac47 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:14:39.809305 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-6bdd429d-e26f-4200-8f3a-e348e9daac47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:14:39.863844 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 88/179] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:14:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 02:14:39.864689 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4241f95f-00f5-4f02-8115-3f6a10353037 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:39.876170 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 92/180] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:14:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 02:14:39.876990 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdd429d-e26f-4200-8f3a-e348e9daac47 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:39.901717 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:39.948427 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:39.969207 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4241f95f-00f5-4f02-8115-3f6a10353037 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:39.989474 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdd429d-e26f-4200-8f3a-e348e9daac47 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:40.003080 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2779471-c3e5-4ff9-bbf8-ee8b2cb74f16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:40.003805 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2779471-c3e5-4ff9-bbf8-ee8b2cb74f16 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:40.004083 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2779471-c3e5-4ff9-bbf8-ee8b2cb74f16 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:40.004354 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2779471-c3e5-4ff9-bbf8-ee8b2cb74f16 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:40.014183 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2779471-c3e5-4ff9-bbf8-ee8b2cb74f16 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 02:14:40.021769 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-f2779471-c3e5-4ff9-bbf8-ee8b2cb74f16 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 02:14:40.021769 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:14:40.021769 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:14:40.021769 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:14:40.021769 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:14:40.021769 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:14:40.021769 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:14:40.021769 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:14:40.021769 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:14:40.021769 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:14:40.021769 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:14:40.021769 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:14:40.021769 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 02:14:40.021769 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 02:14:40.021769 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 02:14:40.021769 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 02:14:40.022765 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 02:14:40.022765 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 02:14:40.022765 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 02:14:40.022765 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 02:14:40.022765 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:14:40.022765 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:14:40.022765 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:14:40.022765 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:14:40.022765 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:14:40.022765 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:14:40.022765 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:14:40.022765 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:14:40.022765 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:14:40.022765 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:14:40.022765 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:14:40.022765 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:14:40.028569 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:14:40.028569 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:14:40.028569 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:14:40.028569 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:14:40.028569 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 02:14:40.028569 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 02:14:40.028569 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 02:14:40.028569 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 02:14:40.028569 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 02:14:40.028569 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 02:14:40.028569 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 02:14:40.028569 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:14:40.029201 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f2f6d0-05d1-4b89-9e74-d714494f795c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:40.029290 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 89/181] 10.222.0.66 () {56 vars in 1096 bytes} [Mon Jul 18 02:14:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:14:40.029360 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f2f6d0-05d1-4b89-9e74-d714494f795c None admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:40.029427 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2779471-c3e5-4ff9-bbf8-ee8b2cb74f16 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:40.029507 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f2f6d0-05d1-4b89-9e74-d714494f795c None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:40.029507 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f2f6d0-05d1-4b89-9e74-d714494f795c None admin] PATH_INFO: `/v3/projects/a5370165a2354ba6aec1bb43138f4838/users/20fb40a9faec4e74bcb9275b58037782/roles/4045191c71164a3e9a009f23ab3b1406` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:40.029507 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f2f6d0-05d1-4b89-9e74-d714494f795c None admin] RBAC: Authorizing `identity:create_grant(project_id=a5370165a2354ba6aec1bb43138f4838, user_id=20fb40a9faec4e74bcb9275b58037782, role_id=4045191c71164a3e9a009f23ab3b1406)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:40.029507 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f2f6d0-05d1-4b89-9e74-d714494f795c None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:40.098174 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 93/182] 10.222.0.66 () {58 vars in 1439 bytes} [Mon Jul 18 02:14:39 2022] PUT /identity/v3/projects/a5370165a2354ba6aec1bb43138f4838/users/20fb40a9faec4e74bcb9275b58037782/roles/4045191c71164a3e9a009f23ab3b1406 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:14:40.098994 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f2f6d0-05d1-4b89-9e74-d714494f795c None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:40.124020 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:40.135883 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2779471-c3e5-4ff9-bbf8-ee8b2cb74f16 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:40.176515 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebac9242-fb62-43d1-80f5-a6f08942ffea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:40.177411 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebac9242-fb62-43d1-80f5-a6f08942ffea None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:40.177688 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebac9242-fb62-43d1-80f5-a6f08942ffea None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:40.177955 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebac9242-fb62-43d1-80f5-a6f08942ffea None admin] PATH_INFO: `/v3/users/62dd67591e0147fe97b248079359a4de` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:40.180020 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebac9242-fb62-43d1-80f5-a6f08942ffea None admin] RBAC: Authorizing `identity:get_user(user_id=62dd67591e0147fe97b248079359a4de)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:40.180721 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebac9242-fb62-43d1-80f5-a6f08942ffea None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:40.182649 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 90/183] 10.222.0.66 () {56 vars in 1177 bytes} [Mon Jul 18 02:14:40 2022] GET /identity/v3/users/62dd67591e0147fe97b248079359a4de => generated 247 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:14:40.183384 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebac9242-fb62-43d1-80f5-a6f08942ffea None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:40.216726 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f2f6d0-05d1-4b89-9e74-d714494f795c None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:40.239968 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:40.258333 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3597425f-60ab-4ba1-a594-c61b452777d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:40.258936 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3597425f-60ab-4ba1-a594-c61b452777d4 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:40.259211 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3597425f-60ab-4ba1-a594-c61b452777d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:40.259490 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3597425f-60ab-4ba1-a594-c61b452777d4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:40.260331 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3597425f-60ab-4ba1-a594-c61b452777d4 None admin] RBAC: Adding query filter params (role.id=65feabf3b27944a8b01a86c8ac608457, scope.project.id=9b769253e8b5410b9c5fabbea39b7078, user.id=8fa31761548948049aa25bd5881382c4) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:14:40.260644 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3597425f-60ab-4ba1-a594-c61b452777d4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:40.262115 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3597425f-60ab-4ba1-a594-c61b452777d4 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:40.269583 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 94/184] 10.222.0.66 () {56 vars in 1374 bytes} [Mon Jul 18 02:14:40 2022] GET /identity/v3/role_assignments?user.id=8fa31761548948049aa25bd5881382c4&scope.project.id=9b769253e8b5410b9c5fabbea39b7078&role.id=65feabf3b27944a8b01a86c8ac608457 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:14:40.269583 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3597425f-60ab-4ba1-a594-c61b452777d4 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:40.318489 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebac9242-fb62-43d1-80f5-a6f08942ffea None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:40.376492 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014426b3-d396-4d2e-a3f2-bf94fb863f83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:40.377435 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014426b3-d396-4d2e-a3f2-bf94fb863f83 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:40.377727 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014426b3-d396-4d2e-a3f2-bf94fb863f83 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:40.378006 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014426b3-d396-4d2e-a3f2-bf94fb863f83 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:40.380023 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014426b3-d396-4d2e-a3f2-bf94fb863f83 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:40.381857 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014426b3-d396-4d2e-a3f2-bf94fb863f83 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:40.384478 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 91/185] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 18 02:14:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:14:40.385584 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014426b3-d396-4d2e-a3f2-bf94fb863f83 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:40.394596 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3597425f-60ab-4ba1-a594-c61b452777d4 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:40.424331 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:40.435809 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1710fd07-0749-423f-aec2-d644af08403d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:40.435809 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1710fd07-0749-423f-aec2-d644af08403d None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:40.436202 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1710fd07-0749-423f-aec2-d644af08403d None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:40.436406 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1710fd07-0749-423f-aec2-d644af08403d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:40.449866 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1710fd07-0749-423f-aec2-d644af08403d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 02:14:40.458967 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-1710fd07-0749-423f-aec2-d644af08403d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 02:14:40.458967 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:14:40.458967 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:14:40.458967 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:14:40.458967 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:14:40.458967 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:14:40.458967 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:14:40.458967 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:14:40.458967 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:14:40.458967 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:14:40.458967 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:14:40.458967 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:14:40.458967 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 02:14:40.458967 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 02:14:40.458967 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 02:14:40.458967 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 02:14:40.459876 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 02:14:40.459876 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 02:14:40.459876 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 02:14:40.459876 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 02:14:40.459876 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:14:40.459876 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:14:40.459876 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:14:40.459876 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:14:40.459876 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:14:40.459876 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:14:40.459876 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:14:40.459876 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:14:40.459876 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:14:40.459876 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:14:40.459876 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:14:40.459876 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:14:40.460687 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:14:40.460687 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:14:40.460687 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:14:40.460687 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:14:40.460687 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 02:14:40.460687 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 02:14:40.460687 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 02:14:40.460687 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 02:14:40.460687 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 02:14:40.460687 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 02:14:40.460687 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 02:14:40.460687 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:14:40.460687 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 95/186] 10.222.0.66 () {56 vars in 1114 bytes} [Mon Jul 18 02:14:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:14:40.461368 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1710fd07-0749-423f-aec2-d644af08403d None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:40.520587 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:40.535230 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014426b3-d396-4d2e-a3f2-bf94fb863f83 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:40.573043 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1959fadc-c0a0-4dbf-aa5c-e1e0157fe061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:40.573734 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1959fadc-c0a0-4dbf-aa5c-e1e0157fe061 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:40.574019 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1959fadc-c0a0-4dbf-aa5c-e1e0157fe061 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:40.574328 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1959fadc-c0a0-4dbf-aa5c-e1e0157fe061 None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:40.575081 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1959fadc-c0a0-4dbf-aa5c-e1e0157fe061 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:14:40.575424 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1959fadc-c0a0-4dbf-aa5c-e1e0157fe061 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:40.576829 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1959fadc-c0a0-4dbf-aa5c-e1e0157fe061 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:40.577651 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1710fd07-0749-423f-aec2-d644af08403d None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:40.588321 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 92/187] 10.222.0.66 () {56 vars in 1099 bytes} [Mon Jul 18 02:14:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:14:40.589235 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1959fadc-c0a0-4dbf-aa5c-e1e0157fe061 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:40.621202 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a376fbc-d3a2-4db4-ada4-bec1e8ac7130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:40.622411 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a376fbc-d3a2-4db4-ada4-bec1e8ac7130 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:40.622748 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a376fbc-d3a2-4db4-ada4-bec1e8ac7130 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:40.623032 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a376fbc-d3a2-4db4-ada4-bec1e8ac7130 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:40.695903 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a376fbc-d3a2-4db4-ada4-bec1e8ac7130 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:40.697393 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a376fbc-d3a2-4db4-ada4-bec1e8ac7130 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:40.697531 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:40.712201 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1959fadc-c0a0-4dbf-aa5c-e1e0157fe061 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:40.742948 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:40.751007 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a871625-5151-4eba-9e2f-6556fa1556dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:40.751687 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a871625-5151-4eba-9e2f-6556fa1556dd None admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:40.755491 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a871625-5151-4eba-9e2f-6556fa1556dd None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:40.755491 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a871625-5151-4eba-9e2f-6556fa1556dd None admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:40.755491 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a871625-5151-4eba-9e2f-6556fa1556dd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:40.755491 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a871625-5151-4eba-9e2f-6556fa1556dd None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:40.804947 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 96/188] 10.222.0.66 () {58 vars in 1282 bytes} [Mon Jul 18 02:14:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 02:14:40.804947 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a376fbc-d3a2-4db4-ada4-bec1e8ac7130 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:40.859596 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 93/189] 10.222.0.66 () {60 vars in 1132 bytes} [Mon Jul 18 02:14:40 2022] POST /identity/v3/users => generated 297 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:14:40.860441 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a871625-5151-4eba-9e2f-6556fa1556dd None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:40.933136 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a376fbc-d3a2-4db4-ada4-bec1e8ac7130 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:40.972822 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55769a3a-b68f-4974-a2d2-0ac937e8848b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:40.973796 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55769a3a-b68f-4974-a2d2-0ac937e8848b None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:40.974374 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55769a3a-b68f-4974-a2d2-0ac937e8848b None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:40.974797 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55769a3a-b68f-4974-a2d2-0ac937e8848b None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:40.975853 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55769a3a-b68f-4974-a2d2-0ac937e8848b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:14:40.976310 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55769a3a-b68f-4974-a2d2-0ac937e8848b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:40.977891 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55769a3a-b68f-4974-a2d2-0ac937e8848b None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:40.987953 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 97/190] 10.222.0.66 () {56 vars in 1111 bytes} [Mon Jul 18 02:14:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:14:40.989334 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55769a3a-b68f-4974-a2d2-0ac937e8848b None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:40.991763 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a871625-5151-4eba-9e2f-6556fa1556dd None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:41.004666 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:41.031110 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1e631e-5226-455c-a8ad-269aa023a7de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:41.032430 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1e631e-5226-455c-a8ad-269aa023a7de None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:41.032983 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1e631e-5226-455c-a8ad-269aa023a7de None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:41.033978 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1e631e-5226-455c-a8ad-269aa023a7de None admin] PATH_INFO: `/v3/users/046d41a707e04cce8aa98b4f4294da80` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:41.050041 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1e631e-5226-455c-a8ad-269aa023a7de None admin] RBAC: Authorizing `identity:get_user(user_id=046d41a707e04cce8aa98b4f4294da80)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:41.051906 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1e631e-5226-455c-a8ad-269aa023a7de None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:41.054251 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 94/191] 10.222.0.66 () {56 vars in 1177 bytes} [Mon Jul 18 02:14:40 2022] GET /identity/v3/users/046d41a707e04cce8aa98b4f4294da80 => generated 283 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:14:41.055336 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1e631e-5226-455c-a8ad-269aa023a7de None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:41.058189 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:41.116524 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55769a3a-b68f-4974-a2d2-0ac937e8848b None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:41.168695 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681aea19-2b3e-4fda-9adb-476e144351f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:41.169566 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681aea19-2b3e-4fda-9adb-476e144351f6 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:41.170303 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681aea19-2b3e-4fda-9adb-476e144351f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:41.170592 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681aea19-2b3e-4fda-9adb-476e144351f6 None admin] PATH_INFO: `/v3/projects/a5370165a2354ba6aec1bb43138f4838` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:41.172060 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681aea19-2b3e-4fda-9adb-476e144351f6 None admin] RBAC: Authorizing `identity:get_project(project_id=a5370165a2354ba6aec1bb43138f4838)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:41.175282 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681aea19-2b3e-4fda-9adb-476e144351f6 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:41.178124 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 98/192] 10.222.0.66 () {56 vars in 1186 bytes} [Mon Jul 18 02:14:40 2022] GET /identity/v3/projects/a5370165a2354ba6aec1bb43138f4838 => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:14:41.178993 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681aea19-2b3e-4fda-9adb-476e144351f6 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:41.184153 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1e631e-5226-455c-a8ad-269aa023a7de None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:41.230293 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79149330-035e-4e6f-b453-f3d7116a6157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:41.231528 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79149330-035e-4e6f-b453-f3d7116a6157 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:41.232036 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79149330-035e-4e6f-b453-f3d7116a6157 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:41.232516 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79149330-035e-4e6f-b453-f3d7116a6157 None admin] PATH_INFO: `/v3/users/046d41a707e04cce8aa98b4f4294da80` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:41.233987 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79149330-035e-4e6f-b453-f3d7116a6157 None admin] RBAC: Authorizing `identity:get_user(user_id=046d41a707e04cce8aa98b4f4294da80)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:41.236198 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79149330-035e-4e6f-b453-f3d7116a6157 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:41.239957 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 95/193] 10.222.0.66 () {56 vars in 1177 bytes} [Mon Jul 18 02:14:41 2022] GET /identity/v3/users/046d41a707e04cce8aa98b4f4294da80 => generated 283 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:14:41.241327 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79149330-035e-4e6f-b453-f3d7116a6157 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:41.280202 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:41.311023 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681aea19-2b3e-4fda-9adb-476e144351f6 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:41.344612 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:41.356622 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393a44ba-5a48-4f02-a257-3be07bacb174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:41.357559 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393a44ba-5a48-4f02-a257-3be07bacb174 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:41.358030 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393a44ba-5a48-4f02-a257-3be07bacb174 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:41.358498 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393a44ba-5a48-4f02-a257-3be07bacb174 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:41.385718 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79149330-035e-4e6f-b453-f3d7116a6157 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:41.429219 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8fd15e-83c3-4afe-b1cb-1d4684637e75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:41.430134 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8fd15e-83c3-4afe-b1cb-1d4684637e75 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:41.430294 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8fd15e-83c3-4afe-b1cb-1d4684637e75 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:41.430577 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8fd15e-83c3-4afe-b1cb-1d4684637e75 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:41.431453 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8fd15e-83c3-4afe-b1cb-1d4684637e75 None admin] RBAC: Adding query filter params (role.id=d4aa3324f0c84887b41d0dc4acf7f30e, scope.project.id=a5370165a2354ba6aec1bb43138f4838, user.id=62dd67591e0147fe97b248079359a4de) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:14:41.431728 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8fd15e-83c3-4afe-b1cb-1d4684637e75 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:41.433190 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8fd15e-83c3-4afe-b1cb-1d4684637e75 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:41.437784 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393a44ba-5a48-4f02-a257-3be07bacb174 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:41.439711 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393a44ba-5a48-4f02-a257-3be07bacb174 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:41.440984 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 96/194] 10.222.0.66 () {56 vars in 1374 bytes} [Mon Jul 18 02:14:41 2022] GET /identity/v3/role_assignments?user.id=62dd67591e0147fe97b248079359a4de&scope.project.id=a5370165a2354ba6aec1bb43138f4838&role.id=d4aa3324f0c84887b41d0dc4acf7f30e => generated 608 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:14:41.441829 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8fd15e-83c3-4afe-b1cb-1d4684637e75 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:41.560874 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8fd15e-83c3-4afe-b1cb-1d4684637e75 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:41.582896 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:41.600288 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eec2ef-cb3a-4ee3-a8b9-378958bd10a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:41.600965 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eec2ef-cb3a-4ee3-a8b9-378958bd10a0 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:41.601250 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eec2ef-cb3a-4ee3-a8b9-378958bd10a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:41.601528 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eec2ef-cb3a-4ee3-a8b9-378958bd10a0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:41.609716 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:41.656618 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 99/195] 10.222.0.66 () {58 vars in 1282 bytes} [Mon Jul 18 02:14:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 478 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 02:14:41.662391 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393a44ba-5a48-4f02-a257-3be07bacb174 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:41.683495 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83eec2ef-cb3a-4ee3-a8b9-378958bd10a0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:41.685110 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83eec2ef-cb3a-4ee3-a8b9-378958bd10a0 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:41.773567 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393a44ba-5a48-4f02-a257-3be07bacb174 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:41.795946 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 97/196] 10.222.0.66 () {58 vars in 1282 bytes} [Mon Jul 18 02:14:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 02:14:41.801916 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eec2ef-cb3a-4ee3-a8b9-378958bd10a0 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:41.810588 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a414b71-4b4d-4dab-b741-0db59c2949b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:41.811806 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a414b71-4b4d-4dab-b741-0db59c2949b4 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:41.812286 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a414b71-4b4d-4dab-b741-0db59c2949b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:41.812766 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a414b71-4b4d-4dab-b741-0db59c2949b4 None admin] PATH_INFO: `/v3/projects/9748772803a44ccea6827637294a3a97` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:41.814191 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a414b71-4b4d-4dab-b741-0db59c2949b4 None admin] RBAC: Authorizing `identity:get_project(project_id=9748772803a44ccea6827637294a3a97)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:41.816503 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a414b71-4b4d-4dab-b741-0db59c2949b4 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:41.818879 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 100/197] 10.222.0.66 () {56 vars in 1186 bytes} [Mon Jul 18 02:14:41 2022] GET /identity/v3/projects/9748772803a44ccea6827637294a3a97 => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 02:14:41.827336 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a414b71-4b4d-4dab-b741-0db59c2949b4 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:41.862525 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:41.915167 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:41.991046 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eec2ef-cb3a-4ee3-a8b9-378958bd10a0 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:42.029116 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc4a899-07f4-4090-9762-d5a5131f0db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:42.029985 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc4a899-07f4-4090-9762-d5a5131f0db0 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:42.032142 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc4a899-07f4-4090-9762-d5a5131f0db0 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:42.032440 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc4a899-07f4-4090-9762-d5a5131f0db0 None admin] PATH_INFO: `/v3/projects/9748772803a44ccea6827637294a3a97` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:42.033838 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc4a899-07f4-4090-9762-d5a5131f0db0 None admin] RBAC: Authorizing `identity:get_project(project_id=9748772803a44ccea6827637294a3a97)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:42.035452 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc4a899-07f4-4090-9762-d5a5131f0db0 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:42.036445 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a414b71-4b4d-4dab-b741-0db59c2949b4 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:42.037384 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 98/198] 10.222.0.66 () {56 vars in 1186 bytes} [Mon Jul 18 02:14:41 2022] GET /identity/v3/projects/9748772803a44ccea6827637294a3a97 => generated 302 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 02:14:42.043032 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc4a899-07f4-4090-9762-d5a5131f0db0 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:42.099343 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d77be4f-2fe4-4f5d-b27e-b1cfaef59d0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:42.100427 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d77be4f-2fe4-4f5d-b27e-b1cfaef59d0a None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:42.100944 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d77be4f-2fe4-4f5d-b27e-b1cfaef59d0a None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:42.101471 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d77be4f-2fe4-4f5d-b27e-b1cfaef59d0a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:42.102586 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d77be4f-2fe4-4f5d-b27e-b1cfaef59d0a None admin] RBAC: Adding query filter params (role.id=d4aa3324f0c84887b41d0dc4acf7f30e, scope.project.id=9748772803a44ccea6827637294a3a97, user.id=046d41a707e04cce8aa98b4f4294da80) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:14:42.103105 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d77be4f-2fe4-4f5d-b27e-b1cfaef59d0a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:42.105020 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d77be4f-2fe4-4f5d-b27e-b1cfaef59d0a None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:42.119284 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 101/199] 10.222.0.66 () {56 vars in 1374 bytes} [Mon Jul 18 02:14:41 2022] GET /identity/v3/role_assignments?user.id=046d41a707e04cce8aa98b4f4294da80&scope.project.id=9748772803a44ccea6827637294a3a97&role.id=d4aa3324f0c84887b41d0dc4acf7f30e => generated 259 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:14:42.162325 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc4a899-07f4-4090-9762-d5a5131f0db0 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:42.163905 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:42.200549 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0de1593-324c-48d3-83b7-e374e1e26e4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:42.201227 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0de1593-324c-48d3-83b7-e374e1e26e4c None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:42.201511 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0de1593-324c-48d3-83b7-e374e1e26e4c None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:42.201786 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0de1593-324c-48d3-83b7-e374e1e26e4c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:42.202668 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0de1593-324c-48d3-83b7-e374e1e26e4c None admin] RBAC: Adding query filter params (role.id=a5b895323e7b4f0db2491abe9ad57ee9, scope.project.id=9748772803a44ccea6827637294a3a97, user.id=046d41a707e04cce8aa98b4f4294da80) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:14:42.202982 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0de1593-324c-48d3-83b7-e374e1e26e4c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:42.204442 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0de1593-324c-48d3-83b7-e374e1e26e4c None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:42.210732 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 99/200] 10.222.0.66 () {56 vars in 1374 bytes} [Mon Jul 18 02:14:42 2022] GET /identity/v3/role_assignments?user.id=046d41a707e04cce8aa98b4f4294da80&scope.project.id=9748772803a44ccea6827637294a3a97&role.id=a5b895323e7b4f0db2491abe9ad57ee9 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 02:14:42.447221 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d13b255-2854-4763-a462-e0c509ffe092 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:42.447574 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d13b255-2854-4763-a462-e0c509ffe092 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:42.447890 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d13b255-2854-4763-a462-e0c509ffe092 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:42.449292 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 102/201] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:14:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:14:42.454549 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1507af99-ac3d-4052-9992-12a3f5af1baa None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:42.455006 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1507af99-ac3d-4052-9992-12a3f5af1baa None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:42.455134 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1507af99-ac3d-4052-9992-12a3f5af1baa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:42.478692 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-1507af99-ac3d-4052-9992-12a3f5af1baa None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:14:42.479893 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-1507af99-ac3d-4052-9992-12a3f5af1baa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:14:42.574275 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 100/202] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:14:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 02:14:42.615694 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d13b255-2854-4763-a462-e0c509ffe092 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:42.750755 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d13b255-2854-4763-a462-e0c509ffe092 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:42.760334 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:42.801191 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bb554a-12d4-413f-85d2-ad87bbebf037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:42.801992 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bb554a-12d4-413f-85d2-ad87bbebf037 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:42.802296 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bb554a-12d4-413f-85d2-ad87bbebf037 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:42.802570 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bb554a-12d4-413f-85d2-ad87bbebf037 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:42.823104 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bb554a-12d4-413f-85d2-ad87bbebf037 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 02:14:42.831606 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-04bb554a-12d4-413f-85d2-ad87bbebf037 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 02:14:42.831606 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:14:42.831606 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:14:42.831606 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:14:42.831606 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:14:42.831606 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:14:42.831606 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:14:42.831606 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:14:42.831606 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:14:42.831606 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:14:42.831606 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:14:42.831606 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:14:42.831606 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 02:14:42.831606 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 02:14:42.831606 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 02:14:42.831606 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 02:14:42.832572 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 02:14:42.832572 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 02:14:42.832572 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 02:14:42.832572 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 02:14:42.832572 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:14:42.832572 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:14:42.832572 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:14:42.832572 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:14:42.832572 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:14:42.832572 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:14:42.832572 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:14:42.832572 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:14:42.832572 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:14:42.832572 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:14:42.832572 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:14:42.832572 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:14:42.833460 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:14:42.833460 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:14:42.833460 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:14:42.833460 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:14:42.833460 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 02:14:42.833460 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 02:14:42.833460 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 02:14:42.833460 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 02:14:42.833460 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 02:14:42.833460 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 02:14:42.833460 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 02:14:42.833460 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:14:42.833460 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 103/203] 10.222.0.66 () {56 vars in 1099 bytes} [Mon Jul 18 02:14:42 2022] GET /identity/v3/roles/reader => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:14:42.838864 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1507af99-ac3d-4052-9992-12a3f5af1baa None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:42.973489 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:42.983268 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1507af99-ac3d-4052-9992-12a3f5af1baa None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:43.027989 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51a20ba-13c8-4814-9fe4-f1fbfca74eaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:43.028573 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51a20ba-13c8-4814-9fe4-f1fbfca74eaf None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:43.028859 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51a20ba-13c8-4814-9fe4-f1fbfca74eaf None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:43.029140 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51a20ba-13c8-4814-9fe4-f1fbfca74eaf None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:43.029895 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51a20ba-13c8-4814-9fe4-f1fbfca74eaf None admin] RBAC: Adding query filter params (name=reader) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:14:43.030236 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51a20ba-13c8-4814-9fe4-f1fbfca74eaf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:43.031756 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51a20ba-13c8-4814-9fe4-f1fbfca74eaf None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:43.044769 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 101/204] 10.222.0.66 () {56 vars in 1101 bytes} [Mon Jul 18 02:14:42 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:14:43.050752 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bb554a-12d4-413f-85d2-ad87bbebf037 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:43.200608 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bb554a-12d4-413f-85d2-ad87bbebf037 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:43.242775 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f15363c-60b3-4303-b265-62587e33f5b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:43.243646 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f15363c-60b3-4303-b265-62587e33f5b5 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:43.243932 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f15363c-60b3-4303-b265-62587e33f5b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:43.244209 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f15363c-60b3-4303-b265-62587e33f5b5 None admin] PATH_INFO: `/v3/users/20fb40a9faec4e74bcb9275b58037782` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:43.246854 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f15363c-60b3-4303-b265-62587e33f5b5 None admin] RBAC: Authorizing `identity:get_user(user_id=20fb40a9faec4e74bcb9275b58037782)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:43.248331 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f15363c-60b3-4303-b265-62587e33f5b5 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:43.250025 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 104/205] 10.222.0.66 () {56 vars in 1177 bytes} [Mon Jul 18 02:14:43 2022] GET /identity/v3/users/20fb40a9faec4e74bcb9275b58037782 => generated 289 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:14:43.256375 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51a20ba-13c8-4814-9fe4-f1fbfca74eaf None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:43.415208 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51a20ba-13c8-4814-9fe4-f1fbfca74eaf None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:43.470320 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f46c83-908b-4296-bbbc-b56bfd4d7261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:43.472600 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f46c83-908b-4296-bbbc-b56bfd4d7261 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:43.473103 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f46c83-908b-4296-bbbc-b56bfd4d7261 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:43.473580 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f46c83-908b-4296-bbbc-b56bfd4d7261 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:43.516998 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:43.582411 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f46c83-908b-4296-bbbc-b56bfd4d7261 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:43.584306 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f46c83-908b-4296-bbbc-b56bfd4d7261 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:43.732155 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 102/206] 10.222.0.66 () {58 vars in 1282 bytes} [Mon Jul 18 02:14:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 02:14:43.737822 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f15363c-60b3-4303-b265-62587e33f5b5 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:43.786425 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:43.853361 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f15363c-60b3-4303-b265-62587e33f5b5 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:43.897691 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f60b5d-fb1e-4807-a20b-eaf5e2eeb166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:43.898605 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f60b5d-fb1e-4807-a20b-eaf5e2eeb166 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:43.898890 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f60b5d-fb1e-4807-a20b-eaf5e2eeb166 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:43.899169 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f60b5d-fb1e-4807-a20b-eaf5e2eeb166 None admin] PATH_INFO: `/v3/projects/a5370165a2354ba6aec1bb43138f4838` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:43.900178 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161e5311-ecdb-48e2-b106-21444c9000df None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:43.900386 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f60b5d-fb1e-4807-a20b-eaf5e2eeb166 None admin] RBAC: Authorizing `identity:get_project(project_id=a5370165a2354ba6aec1bb43138f4838)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:43.900887 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161e5311-ecdb-48e2-b106-21444c9000df None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:43.901364 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161e5311-ecdb-48e2-b106-21444c9000df None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:43.901837 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f60b5d-fb1e-4807-a20b-eaf5e2eeb166 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:43.903604 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 103/207] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:14:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:14:43.907714 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 105/208] 10.222.0.66 () {56 vars in 1186 bytes} [Mon Jul 18 02:14:43 2022] GET /identity/v3/projects/a5370165a2354ba6aec1bb43138f4838 => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:14:43.910027 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f60b5d-fb1e-4807-a20b-eaf5e2eeb166 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:43.910384 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924d4c18-5cbf-4c69-b567-57e02a27ab0f None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:43.910872 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924d4c18-5cbf-4c69-b567-57e02a27ab0f None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:43.911349 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924d4c18-5cbf-4c69-b567-57e02a27ab0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:43.929191 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-924d4c18-5cbf-4c69-b567-57e02a27ab0f None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:14:43.930778 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-924d4c18-5cbf-4c69-b567-57e02a27ab0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:14:44.022841 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:44.050482 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 104/209] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:14:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 02:14:44.050482 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f4630d-f823-4865-a77f-c65d288844a5 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:44.050482 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f4630d-f823-4865-a77f-c65d288844a5 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:44.050482 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f4630d-f823-4865-a77f-c65d288844a5 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:44.050482 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 105/210] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:14:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:14:44.056983 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0edfc2-0d56-4326-b190-fa6c833ade45 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:44.059013 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0edfc2-0d56-4326-b190-fa6c833ade45 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:44.060362 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0edfc2-0d56-4326-b190-fa6c833ade45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:44.084034 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f60b5d-fb1e-4807-a20b-eaf5e2eeb166 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:44.085765 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:44.087379 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-ed0edfc2-0d56-4326-b190-fa6c833ade45 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:14:44.088796 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-ed0edfc2-0d56-4326-b190-fa6c833ade45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:14:44.121487 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2cfe4d-ec03-4b22-82e0-aaa963a160d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:44.122234 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2cfe4d-ec03-4b22-82e0-aaa963a160d3 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:44.122524 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2cfe4d-ec03-4b22-82e0-aaa963a160d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:44.122855 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2cfe4d-ec03-4b22-82e0-aaa963a160d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:44.123726 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2cfe4d-ec03-4b22-82e0-aaa963a160d3 None admin] RBAC: Adding query filter params (role.id=4045191c71164a3e9a009f23ab3b1406, scope.project.id=a5370165a2354ba6aec1bb43138f4838, user.id=20fb40a9faec4e74bcb9275b58037782) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:14:44.124042 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2cfe4d-ec03-4b22-82e0-aaa963a160d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:44.125581 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2cfe4d-ec03-4b22-82e0-aaa963a160d3 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:44.138117 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 106/211] 10.222.0.66 () {56 vars in 1374 bytes} [Mon Jul 18 02:14:43 2022] GET /identity/v3/role_assignments?user.id=20fb40a9faec4e74bcb9275b58037782&scope.project.id=a5370165a2354ba6aec1bb43138f4838&role.id=4045191c71164a3e9a009f23ab3b1406 => generated 608 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 02:14:44.138117 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2cfe4d-ec03-4b22-82e0-aaa963a160d3 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:44.209337 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 106/212] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:14:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 02:14:44.244882 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0edfc2-0d56-4326-b190-fa6c833ade45 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:44.312214 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2cfe4d-ec03-4b22-82e0-aaa963a160d3 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:44.371713 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f6b2cd-c2ac-4a63-b036-bd96981d5fd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:44.372881 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f6b2cd-c2ac-4a63-b036-bd96981d5fd7 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:44.373777 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f6b2cd-c2ac-4a63-b036-bd96981d5fd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:44.374246 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f6b2cd-c2ac-4a63-b036-bd96981d5fd7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:44.375822 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f6b2cd-c2ac-4a63-b036-bd96981d5fd7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:44.378769 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f6b2cd-c2ac-4a63-b036-bd96981d5fd7 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:44.384166 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 107/213] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 18 02:14:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:14:44.393389 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f6b2cd-c2ac-4a63-b036-bd96981d5fd7 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:44.421086 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0edfc2-0d56-4326-b190-fa6c833ade45 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:44.436603 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:44.445056 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:44.467255 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebb8e3e-1417-44b8-86bb-0906559019a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:44.468473 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebb8e3e-1417-44b8-86bb-0906559019a4 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:44.468961 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebb8e3e-1417-44b8-86bb-0906559019a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:44.469558 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebb8e3e-1417-44b8-86bb-0906559019a4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:44.487793 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebb8e3e-1417-44b8-86bb-0906559019a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 02:14:44.497039 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-cebb8e3e-1417-44b8-86bb-0906559019a4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 02:14:44.497039 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:14:44.497039 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:14:44.497039 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:14:44.497039 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:14:44.497039 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:14:44.497039 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:14:44.497039 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:14:44.497039 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:14:44.497039 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:14:44.497039 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:14:44.497039 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:14:44.497039 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 02:14:44.497039 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 02:14:44.497039 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 02:14:44.497039 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 02:14:44.497959 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 02:14:44.497959 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 02:14:44.497959 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 02:14:44.497959 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 02:14:44.497959 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:14:44.497959 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:14:44.497959 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:14:44.497959 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:14:44.497959 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:14:44.497959 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:14:44.497959 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:14:44.497959 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:14:44.497959 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:14:44.497959 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:14:44.497959 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:14:44.497959 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:14:44.499149 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:14:44.499149 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:14:44.499149 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:14:44.499149 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:14:44.499149 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 02:14:44.499149 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 02:14:44.499149 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 02:14:44.499149 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 02:14:44.499149 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 02:14:44.499149 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 02:14:44.499149 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 02:14:44.499149 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:14:44.501709 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 107/214] 10.222.0.66 () {56 vars in 1099 bytes} [Mon Jul 18 02:14:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 02:14:44.508244 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebb8e3e-1417-44b8-86bb-0906559019a4 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:44.524509 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f6b2cd-c2ac-4a63-b036-bd96981d5fd7 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:44.567661 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed0cb8e-6c6a-4ced-8757-555ec7b32096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:44.568494 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed0cb8e-6c6a-4ced-8757-555ec7b32096 None admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:44.569081 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed0cb8e-6c6a-4ced-8757-555ec7b32096 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:44.569467 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed0cb8e-6c6a-4ced-8757-555ec7b32096 None admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:44.570444 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed0cb8e-6c6a-4ced-8757-555ec7b32096 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:44.572017 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed0cb8e-6c6a-4ced-8757-555ec7b32096 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:44.633056 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebb8e3e-1417-44b8-86bb-0906559019a4 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:44.658621 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 108/215] 10.222.0.66 () {60 vars in 1132 bytes} [Mon Jul 18 02:14:44 2022] POST /identity/v3/users => generated 297 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:14:44.660026 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58a63ed-f436-4bfd-bbe2-275453da9bf3 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:44.667604 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58a63ed-f436-4bfd-bbe2-275453da9bf3 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:44.667604 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58a63ed-f436-4bfd-bbe2-275453da9bf3 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:44.667604 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 109/216] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:14:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:14:44.668003 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5847da7c-b360-41f4-bed2-dfea98661610 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:44.668300 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5847da7c-b360-41f4-bed2-dfea98661610 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:44.668585 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5847da7c-b360-41f4-bed2-dfea98661610 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:44.681033 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ff790c-ddfa-405b-a8e7-32416d71bbac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:44.681723 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ff790c-ddfa-405b-a8e7-32416d71bbac None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:44.682005 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ff790c-ddfa-405b-a8e7-32416d71bbac None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:44.682319 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ff790c-ddfa-405b-a8e7-32416d71bbac None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:44.683146 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ff790c-ddfa-405b-a8e7-32416d71bbac None admin] RBAC: Adding query filter params (name=member) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:14:44.683457 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ff790c-ddfa-405b-a8e7-32416d71bbac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:44.684960 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ff790c-ddfa-405b-a8e7-32416d71bbac None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:44.687767 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-5847da7c-b360-41f4-bed2-dfea98661610 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:14:44.689234 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-5847da7c-b360-41f4-bed2-dfea98661610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:14:44.701115 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 108/217] 10.222.0.66 () {56 vars in 1101 bytes} [Mon Jul 18 02:14:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:14:44.703336 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ff790c-ddfa-405b-a8e7-32416d71bbac None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:44.758857 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:44.771389 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 110/218] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:14:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 02:14:44.772837 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a657f1-e2c1-4afd-b211-482f056484ac None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:44.773121 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a657f1-e2c1-4afd-b211-482f056484ac None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:44.773395 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a657f1-e2c1-4afd-b211-482f056484ac None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:44.775088 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 111/219] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:14:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:14:44.780485 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd949d72-5e2a-44fb-a717-b163d35e2e9c None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:44.780773 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd949d72-5e2a-44fb-a717-b163d35e2e9c None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:44.781049 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd949d72-5e2a-44fb-a717-b163d35e2e9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:44.802619 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-cd949d72-5e2a-44fb-a717-b163d35e2e9c None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:14:44.803822 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-cd949d72-5e2a-44fb-a717-b163d35e2e9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:14:44.820402 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:44.838246 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ff790c-ddfa-405b-a8e7-32416d71bbac None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:44.879056 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aca7f26-7e15-4777-9b87-1c6eb9e5438a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:44.879947 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aca7f26-7e15-4777-9b87-1c6eb9e5438a None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:44.880228 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aca7f26-7e15-4777-9b87-1c6eb9e5438a None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:44.880509 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aca7f26-7e15-4777-9b87-1c6eb9e5438a None admin] PATH_INFO: `/v3/users/fdd615f258a64e40b311274fb6639eb4` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:44.885923 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 112/220] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:14:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 02:14:44.886916 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd949d72-5e2a-44fb-a717-b163d35e2e9c None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:44.895413 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aca7f26-7e15-4777-9b87-1c6eb9e5438a None admin] RBAC: Authorizing `identity:get_user(user_id=fdd615f258a64e40b311274fb6639eb4)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:44.896930 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aca7f26-7e15-4777-9b87-1c6eb9e5438a None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:44.898774 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 109/221] 10.222.0.66 () {56 vars in 1177 bytes} [Mon Jul 18 02:14:44 2022] GET /identity/v3/users/fdd615f258a64e40b311274fb6639eb4 => generated 297 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 02:14:44.904273 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aca7f26-7e15-4777-9b87-1c6eb9e5438a None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:45.022296 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd949d72-5e2a-44fb-a717-b163d35e2e9c None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:45.031838 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:45.036009 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aca7f26-7e15-4777-9b87-1c6eb9e5438a None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:45.065276 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac4a250-900f-4c0a-b418-2f6d577b0734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:45.066173 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac4a250-900f-4c0a-b418-2f6d577b0734 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:45.066483 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac4a250-900f-4c0a-b418-2f6d577b0734 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:45.066769 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac4a250-900f-4c0a-b418-2f6d577b0734 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:45.076330 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd89f0c2-744b-43a7-87c7-ece4783a929e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:45.077274 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd89f0c2-744b-43a7-87c7-ece4783a929e None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:45.077762 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd89f0c2-744b-43a7-87c7-ece4783a929e None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:45.078534 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd89f0c2-744b-43a7-87c7-ece4783a929e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:45.079381 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac4a250-900f-4c0a-b418-2f6d577b0734 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 02:14:45.089681 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-4ac4a250-900f-4c0a-b418-2f6d577b0734 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 02:14:45.089681 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:14:45.089681 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:14:45.089681 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:14:45.089681 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:14:45.089681 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:14:45.089681 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:14:45.089681 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:14:45.089681 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:14:45.089681 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:14:45.089681 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:14:45.089681 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:14:45.089681 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 02:14:45.089681 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 02:14:45.089681 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 02:14:45.089681 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 02:14:45.090683 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 02:14:45.090683 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 02:14:45.090683 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 02:14:45.090683 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 02:14:45.090683 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:14:45.090683 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:14:45.090683 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:14:45.090683 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:14:45.090683 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:14:45.090683 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:14:45.090683 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:14:45.090683 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:14:45.090683 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:14:45.090683 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:14:45.090683 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:14:45.090683 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:14:45.091933 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:14:45.091933 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:14:45.091933 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:14:45.091933 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:14:45.091933 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 02:14:45.091933 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 02:14:45.091933 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 02:14:45.091933 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 02:14:45.091933 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 02:14:45.091933 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 02:14:45.091933 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 02:14:45.091933 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:14:45.091933 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 113/222] 10.222.0.66 () {56 vars in 1114 bytes} [Mon Jul 18 02:14:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:14:45.091933 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac4a250-900f-4c0a-b418-2f6d577b0734 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:45.108246 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:45.165372 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd89f0c2-744b-43a7-87c7-ece4783a929e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:45.167428 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd89f0c2-744b-43a7-87c7-ece4783a929e None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:45.224435 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac4a250-900f-4c0a-b418-2f6d577b0734 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:45.262482 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c096e61-dbc9-4d46-9874-7e8b71fc4d7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:45.263277 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c096e61-dbc9-4d46-9874-7e8b71fc4d7e None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:45.263566 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c096e61-dbc9-4d46-9874-7e8b71fc4d7e None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:45.263848 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c096e61-dbc9-4d46-9874-7e8b71fc4d7e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:45.273686 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:45.275550 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c096e61-dbc9-4d46-9874-7e8b71fc4d7e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 02:14:45.279288 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 110/223] 10.222.0.66 () {58 vars in 1282 bytes} [Mon Jul 18 02:14:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 18 02:14:45.280441 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd89f0c2-744b-43a7-87c7-ece4783a929e None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:45.283893 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-0c096e61-dbc9-4d46-9874-7e8b71fc4d7e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 02:14:45.283893 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:14:45.283893 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:14:45.283893 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:14:45.283893 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:14:45.283893 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:14:45.283893 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:14:45.283893 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:14:45.283893 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:14:45.283893 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:14:45.283893 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:14:45.283893 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:14:45.283893 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 02:14:45.283893 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 02:14:45.283893 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 02:14:45.283893 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 02:14:45.288081 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 02:14:45.288081 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 02:14:45.288081 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 02:14:45.288081 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 02:14:45.288081 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:14:45.288081 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:14:45.288081 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:14:45.288081 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:14:45.288081 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:14:45.288081 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:14:45.288081 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:14:45.288081 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:14:45.288081 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:14:45.288081 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:14:45.288081 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:14:45.288081 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:14:45.288902 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:14:45.288902 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:14:45.288902 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:14:45.288902 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:14:45.288902 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 02:14:45.288902 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 02:14:45.288902 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 02:14:45.288902 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 02:14:45.288902 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 02:14:45.288902 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 02:14:45.288902 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 02:14:45.288902 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:14:45.288902 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 114/224] 10.222.0.66 () {56 vars in 1096 bytes} [Mon Jul 18 02:14:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:14:45.288902 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c096e61-dbc9-4d46-9874-7e8b71fc4d7e None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:45.368116 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:45.400753 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c096e61-dbc9-4d46-9874-7e8b71fc4d7e None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:45.404618 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd89f0c2-744b-43a7-87c7-ece4783a929e None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:45.531190 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b520596-141e-4f11-864f-393df1f94ef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:45.531751 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e4b1a2-642c-4880-8046-72bd293d702b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:45.532431 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e4b1a2-642c-4880-8046-72bd293d702b None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:45.532719 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e4b1a2-642c-4880-8046-72bd293d702b None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:45.533536 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e4b1a2-642c-4880-8046-72bd293d702b None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:45.534282 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b520596-141e-4f11-864f-393df1f94ef9 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:45.534403 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e4b1a2-642c-4880-8046-72bd293d702b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:14:45.534682 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e4b1a2-642c-4880-8046-72bd293d702b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:45.535092 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b520596-141e-4f11-864f-393df1f94ef9 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:45.535558 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b520596-141e-4f11-864f-393df1f94ef9 None admin] PATH_INFO: `/v3/projects/9748772803a44ccea6827637294a3a97` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:45.536193 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e4b1a2-642c-4880-8046-72bd293d702b None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:45.537259 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b520596-141e-4f11-864f-393df1f94ef9 None admin] RBAC: Authorizing `identity:get_project(project_id=9748772803a44ccea6827637294a3a97)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:45.539201 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b520596-141e-4f11-864f-393df1f94ef9 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:45.541320 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 115/225] 10.222.0.66 () {56 vars in 1186 bytes} [Mon Jul 18 02:14:45 2022] GET /identity/v3/projects/9748772803a44ccea6827637294a3a97 => generated 302 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:14:45.542484 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b520596-141e-4f11-864f-393df1f94ef9 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:45.545851 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 111/226] 10.222.0.66 () {56 vars in 1111 bytes} [Mon Jul 18 02:14:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:14:45.551342 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e4b1a2-642c-4880-8046-72bd293d702b None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:45.619047 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:45.660416 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b520596-141e-4f11-864f-393df1f94ef9 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:45.670341 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e4b1a2-642c-4880-8046-72bd293d702b None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:45.697698 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24a0847-3e0e-4d15-8533-e93061606c4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:45.698867 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24a0847-3e0e-4d15-8533-e93061606c4e None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:45.699380 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24a0847-3e0e-4d15-8533-e93061606c4e None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:45.699851 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24a0847-3e0e-4d15-8533-e93061606c4e None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:45.700946 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24a0847-3e0e-4d15-8533-e93061606c4e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:14:45.701492 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24a0847-3e0e-4d15-8533-e93061606c4e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:45.703144 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24a0847-3e0e-4d15-8533-e93061606c4e None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:45.707209 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b4f920-8760-45b2-8217-60c9a22e4efe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:45.707888 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b4f920-8760-45b2-8217-60c9a22e4efe None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:45.708177 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b4f920-8760-45b2-8217-60c9a22e4efe None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:45.708461 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b4f920-8760-45b2-8217-60c9a22e4efe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:45.709294 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b4f920-8760-45b2-8217-60c9a22e4efe None admin] RBAC: Adding query filter params (role.id=65feabf3b27944a8b01a86c8ac608457, scope.project.id=9748772803a44ccea6827637294a3a97, user.id=fdd615f258a64e40b311274fb6639eb4) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:14:45.709610 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b4f920-8760-45b2-8217-60c9a22e4efe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:45.710372 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:45.711902 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b4f920-8760-45b2-8217-60c9a22e4efe None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:45.715139 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 116/227] 10.222.0.66 () {56 vars in 1099 bytes} [Mon Jul 18 02:14:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:14:45.718235 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 112/228] 10.222.0.66 () {56 vars in 1374 bytes} [Mon Jul 18 02:14:45 2022] GET /identity/v3/role_assignments?user.id=fdd615f258a64e40b311274fb6639eb4&scope.project.id=9748772803a44ccea6827637294a3a97&role.id=65feabf3b27944a8b01a86c8ac608457 => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:14:45.720559 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24a0847-3e0e-4d15-8533-e93061606c4e None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:45.721316 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b4f920-8760-45b2-8217-60c9a22e4efe None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:45.836884 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b4f920-8760-45b2-8217-60c9a22e4efe None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:45.837758 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24a0847-3e0e-4d15-8533-e93061606c4e None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:45.874637 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:45.877090 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1035963b-8008-4ba3-a6f6-b7b7e14787ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:45.878210 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1035963b-8008-4ba3-a6f6-b7b7e14787ed None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:45.878580 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1035963b-8008-4ba3-a6f6-b7b7e14787ed None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:45.878991 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1035963b-8008-4ba3-a6f6-b7b7e14787ed None admin] PATH_INFO: `/v3/users/046d41a707e04cce8aa98b4f4294da80` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:45.880300 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1035963b-8008-4ba3-a6f6-b7b7e14787ed None admin] RBAC: Authorizing `identity:get_user(user_id=046d41a707e04cce8aa98b4f4294da80)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:45.881892 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1035963b-8008-4ba3-a6f6-b7b7e14787ed None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:45.887990 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 117/229] 10.222.0.66 () {56 vars in 1177 bytes} [Mon Jul 18 02:14:45 2022] GET /identity/v3/users/046d41a707e04cce8aa98b4f4294da80 => generated 283 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:14:45.888664 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf397436-9dc3-491d-b391-3a6e7e53a14e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:45.889823 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf397436-9dc3-491d-b391-3a6e7e53a14e None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:45.890347 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf397436-9dc3-491d-b391-3a6e7e53a14e None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:45.890785 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf397436-9dc3-491d-b391-3a6e7e53a14e None admin] PATH_INFO: `/v3/users/046d41a707e04cce8aa98b4f4294da80` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:45.893178 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf397436-9dc3-491d-b391-3a6e7e53a14e None admin] RBAC: Authorizing `identity:get_user(user_id=046d41a707e04cce8aa98b4f4294da80)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:45.895034 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf397436-9dc3-491d-b391-3a6e7e53a14e None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:45.895850 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1035963b-8008-4ba3-a6f6-b7b7e14787ed None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:45.897536 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 113/230] 10.222.0.66 () {56 vars in 1177 bytes} [Mon Jul 18 02:14:45 2022] GET /identity/v3/users/046d41a707e04cce8aa98b4f4294da80 => generated 283 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:14:45.920123 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf397436-9dc3-491d-b391-3a6e7e53a14e None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:45.978641 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:46.013615 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1035963b-8008-4ba3-a6f6-b7b7e14787ed None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:46.058183 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b176efb4-9a5e-49db-b7fd-75c7c314eb07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:46.059059 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b176efb4-9a5e-49db-b7fd-75c7c314eb07 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:46.059340 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b176efb4-9a5e-49db-b7fd-75c7c314eb07 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:46.059616 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b176efb4-9a5e-49db-b7fd-75c7c314eb07 None admin] PATH_INFO: `/v3/projects/9748772803a44ccea6827637294a3a97` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:46.061012 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b176efb4-9a5e-49db-b7fd-75c7c314eb07 None admin] RBAC: Authorizing `identity:get_project(project_id=9748772803a44ccea6827637294a3a97)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:46.062310 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b176efb4-9a5e-49db-b7fd-75c7c314eb07 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:46.064220 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 118/231] 10.222.0.66 () {56 vars in 1186 bytes} [Mon Jul 18 02:14:45 2022] GET /identity/v3/projects/9748772803a44ccea6827637294a3a97 => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:14:46.070257 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b176efb4-9a5e-49db-b7fd-75c7c314eb07 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:46.074242 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf397436-9dc3-491d-b391-3a6e7e53a14e None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:46.114961 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bddee4-7667-4e38-9a44-6a90f6cbe621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:46.116216 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bddee4-7667-4e38-9a44-6a90f6cbe621 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:46.116727 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bddee4-7667-4e38-9a44-6a90f6cbe621 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:46.117206 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bddee4-7667-4e38-9a44-6a90f6cbe621 None admin] PATH_INFO: `/v3/projects/9748772803a44ccea6827637294a3a97` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:46.119033 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bddee4-7667-4e38-9a44-6a90f6cbe621 None admin] RBAC: Authorizing `identity:get_project(project_id=9748772803a44ccea6827637294a3a97)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:46.120796 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bddee4-7667-4e38-9a44-6a90f6cbe621 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:46.123422 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 114/232] 10.222.0.66 () {56 vars in 1186 bytes} [Mon Jul 18 02:14:45 2022] GET /identity/v3/projects/9748772803a44ccea6827637294a3a97 => generated 302 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:14:46.128990 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bddee4-7667-4e38-9a44-6a90f6cbe621 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:46.208043 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:46.256538 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bddee4-7667-4e38-9a44-6a90f6cbe621 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:46.281031 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b176efb4-9a5e-49db-b7fd-75c7c314eb07 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:46.294950 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42859e23-c58b-4423-8c82-c5ee5df67444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:46.296322 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42859e23-c58b-4423-8c82-c5ee5df67444 None admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:46.296813 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42859e23-c58b-4423-8c82-c5ee5df67444 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:46.297314 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42859e23-c58b-4423-8c82-c5ee5df67444 None admin] PATH_INFO: `/v3/projects/9748772803a44ccea6827637294a3a97/users/046d41a707e04cce8aa98b4f4294da80/roles/d4aa3324f0c84887b41d0dc4acf7f30e` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:46.300698 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42859e23-c58b-4423-8c82-c5ee5df67444 None admin] RBAC: Authorizing `identity:create_grant(project_id=9748772803a44ccea6827637294a3a97, user_id=046d41a707e04cce8aa98b4f4294da80, role_id=d4aa3324f0c84887b41d0dc4acf7f30e)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:46.302419 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42859e23-c58b-4423-8c82-c5ee5df67444 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:46.317542 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925ddadf-da2a-4903-b602-8e6156c22450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:46.320513 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925ddadf-da2a-4903-b602-8e6156c22450 None admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:46.320513 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925ddadf-da2a-4903-b602-8e6156c22450 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:46.320513 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925ddadf-da2a-4903-b602-8e6156c22450 None admin] PATH_INFO: `/v3/projects/9748772803a44ccea6827637294a3a97/users/046d41a707e04cce8aa98b4f4294da80/roles/a5b895323e7b4f0db2491abe9ad57ee9` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:46.322696 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925ddadf-da2a-4903-b602-8e6156c22450 None admin] RBAC: Authorizing `identity:create_grant(project_id=9748772803a44ccea6827637294a3a97, user_id=046d41a707e04cce8aa98b4f4294da80, role_id=a5b895323e7b4f0db2491abe9ad57ee9)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:46.324246 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925ddadf-da2a-4903-b602-8e6156c22450 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:46.327898 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:46.386668 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 115/233] 10.222.0.66 () {58 vars in 1439 bytes} [Mon Jul 18 02:14:46 2022] PUT /identity/v3/projects/9748772803a44ccea6827637294a3a97/users/046d41a707e04cce8aa98b4f4294da80/roles/d4aa3324f0c84887b41d0dc4acf7f30e => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:14:46.431943 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 119/234] 10.222.0.66 () {58 vars in 1439 bytes} [Mon Jul 18 02:14:46 2022] PUT /identity/v3/projects/9748772803a44ccea6827637294a3a97/users/046d41a707e04cce8aa98b4f4294da80/roles/a5b895323e7b4f0db2491abe9ad57ee9 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 02:14:46.994398 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86a7713-4bf1-4661-82a1-071f6bf65c07 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:46.994929 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86a7713-4bf1-4661-82a1-071f6bf65c07 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:46.994929 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86a7713-4bf1-4661-82a1-071f6bf65c07 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:46.996605 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 116/235] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:14:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:14:47.002823 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d62d16-21f1-4adb-b668-956641b88bee None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:47.003248 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d62d16-21f1-4adb-b668-956641b88bee None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:47.003390 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d62d16-21f1-4adb-b668-956641b88bee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:47.021517 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-97d62d16-21f1-4adb-b668-956641b88bee None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:14:47.022722 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-97d62d16-21f1-4adb-b668-956641b88bee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:14:47.112625 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 120/236] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:14:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 02:14:47.148481 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86a7713-4bf1-4661-82a1-071f6bf65c07 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:47.245849 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3b05c2-c277-40f7-a674-24e4ac809e47 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:47.246040 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3b05c2-c277-40f7-a674-24e4ac809e47 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:47.246356 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3b05c2-c277-40f7-a674-24e4ac809e47 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:47.247759 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 121/237] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:14:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 02:14:47.253648 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c851b581-9f01-421f-8a9d-5cf080212b66 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:47.253936 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c851b581-9f01-421f-8a9d-5cf080212b66 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:47.254290 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c851b581-9f01-421f-8a9d-5cf080212b66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:47.284200 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-c851b581-9f01-421f-8a9d-5cf080212b66 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:14:47.285406 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-c851b581-9f01-421f-8a9d-5cf080212b66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:14:47.306378 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86a7713-4bf1-4661-82a1-071f6bf65c07 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:47.342626 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:47.370032 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1131eb-5dd9-4291-bd42-8a1dbcf2fd15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:47.371908 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1131eb-5dd9-4291-bd42-8a1dbcf2fd15 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:47.371908 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1131eb-5dd9-4291-bd42-8a1dbcf2fd15 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:47.371908 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1131eb-5dd9-4291-bd42-8a1dbcf2fd15 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:47.393358 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1131eb-5dd9-4291-bd42-8a1dbcf2fd15 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 02:14:47.404083 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-ba1131eb-5dd9-4291-bd42-8a1dbcf2fd15 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 02:14:47.404083 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:14:47.404083 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:14:47.404083 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:14:47.404083 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:14:47.404083 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:14:47.404083 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:14:47.404083 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:14:47.404083 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:14:47.404083 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:14:47.404083 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:14:47.404083 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:14:47.404083 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 02:14:47.404083 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 02:14:47.404083 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 02:14:47.404083 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 02:14:47.407234 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 02:14:47.407234 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 02:14:47.407234 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 02:14:47.407234 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 02:14:47.407234 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:14:47.407234 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:14:47.407234 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:14:47.407234 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:14:47.407234 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:14:47.407234 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:14:47.407234 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:14:47.407234 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:14:47.407234 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:14:47.407234 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:14:47.407234 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:14:47.407234 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:14:47.408370 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:14:47.408370 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:14:47.408370 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:14:47.408370 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:14:47.408370 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 02:14:47.408370 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 02:14:47.408370 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 02:14:47.408370 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 02:14:47.408370 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 02:14:47.408370 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 02:14:47.408370 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 02:14:47.408370 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:14:47.408370 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 117/238] 10.222.0.66 () {56 vars in 1099 bytes} [Mon Jul 18 02:14:47 2022] GET /identity/v3/roles/reader => generated 84 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:14:47.412582 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1131eb-5dd9-4291-bd42-8a1dbcf2fd15 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:47.418807 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 122/239] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:14:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 02:14:47.475999 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c851b581-9f01-421f-8a9d-5cf080212b66 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:47.588732 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:47.600649 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1131eb-5dd9-4291-bd42-8a1dbcf2fd15 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:47.651244 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4412c8-c5af-4079-a70c-d38032d306ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:47.651925 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4412c8-c5af-4079-a70c-d38032d306ce None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:47.652212 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4412c8-c5af-4079-a70c-d38032d306ce None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:47.652491 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4412c8-c5af-4079-a70c-d38032d306ce None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:47.656211 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4412c8-c5af-4079-a70c-d38032d306ce None admin] RBAC: Adding query filter params (name=reader) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:14:47.656211 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4412c8-c5af-4079-a70c-d38032d306ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:47.656211 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4412c8-c5af-4079-a70c-d38032d306ce None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:47.665587 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 118/240] 10.222.0.66 () {56 vars in 1101 bytes} [Mon Jul 18 02:14:47 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 02:14:47.671403 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4412c8-c5af-4079-a70c-d38032d306ce None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:47.672641 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c851b581-9f01-421f-8a9d-5cf080212b66 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:47.721655 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:47.747066 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a5690e-38c1-4d6f-842a-ed110dc7eeb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:47.748076 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a5690e-38c1-4d6f-842a-ed110dc7eeb3 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:47.748364 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a5690e-38c1-4d6f-842a-ed110dc7eeb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:47.748644 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a5690e-38c1-4d6f-842a-ed110dc7eeb3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:47.753436 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a5690e-38c1-4d6f-842a-ed110dc7eeb3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:47.755059 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a5690e-38c1-4d6f-842a-ed110dc7eeb3 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:47.756772 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 123/241] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 18 02:14:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 02:14:47.767433 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a5690e-38c1-4d6f-842a-ed110dc7eeb3 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:47.814027 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4412c8-c5af-4079-a70c-d38032d306ce None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:47.884769 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1699283a-7e1f-4f23-bd97-ef2014b45ef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:47.885579 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1699283a-7e1f-4f23-bd97-ef2014b45ef5 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:47.885865 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1699283a-7e1f-4f23-bd97-ef2014b45ef5 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:47.886171 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1699283a-7e1f-4f23-bd97-ef2014b45ef5 None admin] PATH_INFO: `/v3/users/a6a181b5b3cf45b9b43ec3a81222b2c7` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:47.917003 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1699283a-7e1f-4f23-bd97-ef2014b45ef5 None admin] RBAC: Authorizing `identity:get_user(user_id=a6a181b5b3cf45b9b43ec3a81222b2c7)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:47.918530 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1699283a-7e1f-4f23-bd97-ef2014b45ef5 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:47.920291 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 119/242] 10.222.0.66 () {56 vars in 1177 bytes} [Mon Jul 18 02:14:47 2022] GET /identity/v3/users/a6a181b5b3cf45b9b43ec3a81222b2c7 => generated 297 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:14:47.931106 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1699283a-7e1f-4f23-bd97-ef2014b45ef5 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:47.950864 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a5690e-38c1-4d6f-842a-ed110dc7eeb3 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:47.976928 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:47.988238 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a0e58e-719b-4919-974a-476adf181b08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:47.988974 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a0e58e-719b-4919-974a-476adf181b08 None admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:47.989264 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a0e58e-719b-4919-974a-476adf181b08 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:47.990436 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a0e58e-719b-4919-974a-476adf181b08 None admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:47.990436 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a0e58e-719b-4919-974a-476adf181b08 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:47.992503 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a0e58e-719b-4919-974a-476adf181b08 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:48.078208 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 124/243] 10.222.0.66 () {60 vars in 1132 bytes} [Mon Jul 18 02:14:47 2022] POST /identity/v3/users => generated 293 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 02:14:48.081667 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:48.090422 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1699283a-7e1f-4f23-bd97-ef2014b45ef5 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:48.137684 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4851de3-88c3-4b42-8ea8-665f48f26b06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:48.138402 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4851de3-88c3-4b42-8ea8-665f48f26b06 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:48.138697 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4851de3-88c3-4b42-8ea8-665f48f26b06 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:48.138977 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4851de3-88c3-4b42-8ea8-665f48f26b06 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:48.234854 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4851de3-88c3-4b42-8ea8-665f48f26b06 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:48.237136 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4851de3-88c3-4b42-8ea8-665f48f26b06 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:48.255446 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821a74fb-a1f4-4125-a4ea-50783b59c27b None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:48.255751 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821a74fb-a1f4-4125-a4ea-50783b59c27b None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:48.256015 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821a74fb-a1f4-4125-a4ea-50783b59c27b None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:48.257441 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 125/244] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:14:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 02:14:48.263055 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42038ee1-8218-4772-8071-ae0375ea973e None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:48.263333 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42038ee1-8218-4772-8071-ae0375ea973e None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:48.263996 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42038ee1-8218-4772-8071-ae0375ea973e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:48.303270 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-42038ee1-8218-4772-8071-ae0375ea973e None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:14:48.304497 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-42038ee1-8218-4772-8071-ae0375ea973e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:14:48.399098 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:48.403584 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 120/245] 10.222.0.66 () {58 vars in 1282 bytes} [Mon Jul 18 02:14:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 473 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 02:14:48.409900 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4851de3-88c3-4b42-8ea8-665f48f26b06 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:48.422590 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 126/246] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:14:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 02:14:48.447594 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42038ee1-8218-4772-8071-ae0375ea973e None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:48.536298 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4851de3-88c3-4b42-8ea8-665f48f26b06 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:48.559478 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42038ee1-8218-4772-8071-ae0375ea973e None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:48.572363 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:48.582944 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5d40fa-e32c-49eb-b2f3-0a6e541f495b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:48.584173 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5d40fa-e32c-49eb-b2f3-0a6e541f495b None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:48.584655 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5d40fa-e32c-49eb-b2f3-0a6e541f495b None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:48.585150 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5d40fa-e32c-49eb-b2f3-0a6e541f495b None admin] PATH_INFO: `/v3/projects/9748772803a44ccea6827637294a3a97` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:48.587319 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5d40fa-e32c-49eb-b2f3-0a6e541f495b None admin] RBAC: Authorizing `identity:get_project(project_id=9748772803a44ccea6827637294a3a97)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:48.589002 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5d40fa-e32c-49eb-b2f3-0a6e541f495b None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:48.591024 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 121/247] 10.222.0.66 () {56 vars in 1186 bytes} [Mon Jul 18 02:14:48 2022] GET /identity/v3/projects/9748772803a44ccea6827637294a3a97 => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:14:48.598371 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5d40fa-e32c-49eb-b2f3-0a6e541f495b None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:48.598701 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9009cb55-699b-4b15-85e5-812a73ae657f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:48.599687 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9009cb55-699b-4b15-85e5-812a73ae657f None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:48.600009 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9009cb55-699b-4b15-85e5-812a73ae657f None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:48.600341 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9009cb55-699b-4b15-85e5-812a73ae657f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:48.612751 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9009cb55-699b-4b15-85e5-812a73ae657f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 02:14:48.623055 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-9009cb55-699b-4b15-85e5-812a73ae657f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 02:14:48.623055 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:14:48.623055 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:14:48.623055 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:14:48.623055 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:14:48.623055 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:14:48.623055 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:14:48.623055 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:14:48.623055 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:14:48.623055 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:14:48.623055 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:14:48.623055 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:14:48.623055 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 02:14:48.623055 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 02:14:48.623055 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 02:14:48.623055 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 02:14:48.624225 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 02:14:48.624225 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 02:14:48.624225 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 02:14:48.624225 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 02:14:48.624225 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:14:48.624225 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:14:48.624225 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:14:48.624225 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:14:48.624225 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:14:48.624225 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:14:48.624225 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:14:48.624225 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:14:48.624225 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:14:48.624225 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:14:48.624225 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:14:48.624225 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:14:48.625304 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:14:48.625304 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:14:48.625304 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:14:48.625304 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:14:48.625304 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 02:14:48.625304 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 02:14:48.625304 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 02:14:48.625304 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 02:14:48.625304 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 02:14:48.625304 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 02:14:48.625304 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 02:14:48.625304 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:14:48.630137 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 127/248] 10.222.0.66 () {56 vars in 1099 bytes} [Mon Jul 18 02:14:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 02:14:48.640287 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9009cb55-699b-4b15-85e5-812a73ae657f None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:48.714460 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:48.728540 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5d40fa-e32c-49eb-b2f3-0a6e541f495b None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:48.757002 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9009cb55-699b-4b15-85e5-812a73ae657f None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:48.767057 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cd104b-3691-4ace-9066-b48e8528e3c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:48.767813 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cd104b-3691-4ace-9066-b48e8528e3c0 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:48.768114 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cd104b-3691-4ace-9066-b48e8528e3c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:48.768399 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cd104b-3691-4ace-9066-b48e8528e3c0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:48.769536 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cd104b-3691-4ace-9066-b48e8528e3c0 None admin] RBAC: Adding query filter params (role.id=4045191c71164a3e9a009f23ab3b1406, scope.project.id=9748772803a44ccea6827637294a3a97, user.id=a6a181b5b3cf45b9b43ec3a81222b2c7) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:14:48.769862 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cd104b-3691-4ace-9066-b48e8528e3c0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:48.771899 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cd104b-3691-4ace-9066-b48e8528e3c0 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:48.779098 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 122/249] 10.222.0.66 () {56 vars in 1374 bytes} [Mon Jul 18 02:14:48 2022] GET /identity/v3/role_assignments?user.id=a6a181b5b3cf45b9b43ec3a81222b2c7&scope.project.id=9748772803a44ccea6827637294a3a97&role.id=4045191c71164a3e9a009f23ab3b1406 => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:14:48.795283 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc825449-d731-489c-84b2-0a16bf2e06bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:48.796083 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc825449-d731-489c-84b2-0a16bf2e06bb None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:48.796372 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc825449-d731-489c-84b2-0a16bf2e06bb None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:48.796650 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc825449-d731-489c-84b2-0a16bf2e06bb None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:48.797441 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc825449-d731-489c-84b2-0a16bf2e06bb None admin] RBAC: Adding query filter params (name=member) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:14:48.797752 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc825449-d731-489c-84b2-0a16bf2e06bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:48.800277 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc825449-d731-489c-84b2-0a16bf2e06bb None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:48.810826 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 128/250] 10.222.0.66 () {56 vars in 1101 bytes} [Mon Jul 18 02:14:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:14:48.816767 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cd104b-3691-4ace-9066-b48e8528e3c0 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:48.828976 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4944d1d5-8e25-446d-8708-260376cd573b None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:48.829305 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4944d1d5-8e25-446d-8708-260376cd573b None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:48.829585 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4944d1d5-8e25-446d-8708-260376cd573b None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:48.831196 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 129/251] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:14:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 02:14:48.836758 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f656f56-635c-4834-8366-99b9bd278e19 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:48.837046 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f656f56-635c-4834-8366-99b9bd278e19 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:48.837325 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f656f56-635c-4834-8366-99b9bd278e19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:48.855408 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-2f656f56-635c-4834-8366-99b9bd278e19 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:14:48.856734 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-2f656f56-635c-4834-8366-99b9bd278e19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:14:48.887145 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:48.937667 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cd104b-3691-4ace-9066-b48e8528e3c0 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:48.937894 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 130/252] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:14:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 02:14:48.963114 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f656f56-635c-4834-8366-99b9bd278e19 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:48.974526 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:48.979034 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec0e0c4-04e2-4e78-a66e-3a04f2ec8522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:48.979928 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec0e0c4-04e2-4e78-a66e-3a04f2ec8522 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:48.980217 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec0e0c4-04e2-4e78-a66e-3a04f2ec8522 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:48.980493 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec0e0c4-04e2-4e78-a66e-3a04f2ec8522 None admin] PATH_INFO: `/v3/users/fdd615f258a64e40b311274fb6639eb4` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:48.983320 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec0e0c4-04e2-4e78-a66e-3a04f2ec8522 None admin] RBAC: Authorizing `identity:get_user(user_id=fdd615f258a64e40b311274fb6639eb4)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:48.984981 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec0e0c4-04e2-4e78-a66e-3a04f2ec8522 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:48.987756 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 123/253] 10.222.0.66 () {56 vars in 1177 bytes} [Mon Jul 18 02:14:48 2022] GET /identity/v3/users/fdd615f258a64e40b311274fb6639eb4 => generated 297 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:14:48.993612 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec0e0c4-04e2-4e78-a66e-3a04f2ec8522 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:49.077601 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f656f56-635c-4834-8366-99b9bd278e19 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:49.106805 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec0e0c4-04e2-4e78-a66e-3a04f2ec8522 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:49.114535 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60231916-2285-4acd-9634-f1447951b058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:49.115349 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60231916-2285-4acd-9634-f1447951b058 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:49.115638 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60231916-2285-4acd-9634-f1447951b058 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:49.115913 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60231916-2285-4acd-9634-f1447951b058 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:49.127112 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60231916-2285-4acd-9634-f1447951b058 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 02:14:49.135550 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-60231916-2285-4acd-9634-f1447951b058 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 02:14:49.135550 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:14:49.135550 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:14:49.135550 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:14:49.135550 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:14:49.135550 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:14:49.135550 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:14:49.135550 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:14:49.135550 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:14:49.135550 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:14:49.135550 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:14:49.135550 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:14:49.135550 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 02:14:49.135550 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 02:14:49.135550 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 02:14:49.135550 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 02:14:49.136460 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 02:14:49.136460 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 02:14:49.136460 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 02:14:49.136460 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 02:14:49.136460 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:14:49.136460 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:14:49.136460 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:14:49.136460 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:14:49.136460 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:14:49.136460 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:14:49.136460 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:14:49.136460 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:14:49.136460 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:14:49.136460 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:14:49.136460 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:14:49.136460 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:14:49.137259 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:14:49.137259 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:14:49.137259 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:14:49.137259 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:14:49.137259 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 02:14:49.137259 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 02:14:49.137259 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 02:14:49.137259 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 02:14:49.137259 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 02:14:49.137259 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 02:14:49.137259 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 02:14:49.137259 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:14:49.137259 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 131/254] 10.222.0.66 () {56 vars in 1096 bytes} [Mon Jul 18 02:14:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 02:14:49.138431 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89be56b-f487-40c7-b1cc-dc9cf6bde484 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:49.142490 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89be56b-f487-40c7-b1cc-dc9cf6bde484 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:49.142858 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89be56b-f487-40c7-b1cc-dc9cf6bde484 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:49.144220 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 132/255] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:14:49 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:14:49.144953 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89be56b-f487-40c7-b1cc-dc9cf6bde484 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:49.149018 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c995383-55f3-4593-976e-bb8e158b3537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:49.150746 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c995383-55f3-4593-976e-bb8e158b3537 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:49.150746 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c995383-55f3-4593-976e-bb8e158b3537 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:49.150746 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c995383-55f3-4593-976e-bb8e158b3537 None admin] PATH_INFO: `/v3/projects/9748772803a44ccea6827637294a3a97` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:49.151829 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c995383-55f3-4593-976e-bb8e158b3537 None admin] RBAC: Authorizing `identity:get_project(project_id=9748772803a44ccea6827637294a3a97)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:49.153311 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c995383-55f3-4593-976e-bb8e158b3537 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:49.155362 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 124/256] 10.222.0.66 () {56 vars in 1186 bytes} [Mon Jul 18 02:14:48 2022] GET /identity/v3/projects/9748772803a44ccea6827637294a3a97 => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 02:14:49.156527 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411d47cd-dbe6-4784-bf55-2c115d0db3b8 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:49.156813 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411d47cd-dbe6-4784-bf55-2c115d0db3b8 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:49.157091 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411d47cd-dbe6-4784-bf55-2c115d0db3b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:49.176978 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:49.181048 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-411d47cd-dbe6-4784-bf55-2c115d0db3b8 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:14:49.182500 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-411d47cd-dbe6-4784-bf55-2c115d0db3b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:14:49.255902 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 125/257] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:14:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 02:14:49.256762 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411d47cd-dbe6-4784-bf55-2c115d0db3b8 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:49.257320 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:49.263368 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89be56b-f487-40c7-b1cc-dc9cf6bde484 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:49.302246 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de57826-5e63-4cc5-a1e8-9cbfd5553f78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:49.303302 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de57826-5e63-4cc5-a1e8-9cbfd5553f78 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:49.303792 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de57826-5e63-4cc5-a1e8-9cbfd5553f78 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:49.304291 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de57826-5e63-4cc5-a1e8-9cbfd5553f78 None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:49.305294 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de57826-5e63-4cc5-a1e8-9cbfd5553f78 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:14:49.305804 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de57826-5e63-4cc5-a1e8-9cbfd5553f78 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:49.307498 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de57826-5e63-4cc5-a1e8-9cbfd5553f78 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:49.317706 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 133/258] 10.222.0.66 () {56 vars in 1099 bytes} [Mon Jul 18 02:14:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:14:49.322955 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de57826-5e63-4cc5-a1e8-9cbfd5553f78 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:49.365811 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411d47cd-dbe6-4784-bf55-2c115d0db3b8 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:49.408697 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922e1630-5035-4256-9f3c-aa1269849c8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:49.410533 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922e1630-5035-4256-9f3c-aa1269849c8f None admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:49.411344 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922e1630-5035-4256-9f3c-aa1269849c8f None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:49.412141 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922e1630-5035-4256-9f3c-aa1269849c8f None admin] PATH_INFO: `/v3/projects/9748772803a44ccea6827637294a3a97/users/fdd615f258a64e40b311274fb6639eb4/roles/65feabf3b27944a8b01a86c8ac608457` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:49.415118 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922e1630-5035-4256-9f3c-aa1269849c8f None admin] RBAC: Authorizing `identity:create_grant(project_id=9748772803a44ccea6827637294a3a97, user_id=fdd615f258a64e40b311274fb6639eb4, role_id=65feabf3b27944a8b01a86c8ac608457)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:49.416923 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:49.417325 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922e1630-5035-4256-9f3c-aa1269849c8f None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:49.439378 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de57826-5e63-4cc5-a1e8-9cbfd5553f78 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:49.477806 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcd66fa-3fc2-4350-a4d9-dce5ed86d954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:49.481154 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcd66fa-3fc2-4350-a4d9-dce5ed86d954 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:49.483363 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcd66fa-3fc2-4350-a4d9-dce5ed86d954 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:49.486958 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 126/259] 10.222.0.66 () {58 vars in 1439 bytes} [Mon Jul 18 02:14:49 2022] PUT /identity/v3/projects/9748772803a44ccea6827637294a3a97/users/fdd615f258a64e40b311274fb6639eb4/roles/65feabf3b27944a8b01a86c8ac608457 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:14:49.487943 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922e1630-5035-4256-9f3c-aa1269849c8f None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:49.488929 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcd66fa-3fc2-4350-a4d9-dce5ed86d954 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:49.514880 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcd66fa-3fc2-4350-a4d9-dce5ed86d954 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 02:14:49.534529 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-adcd66fa-3fc2-4350-a4d9-dce5ed86d954 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 02:14:49.534529 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:14:49.534529 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:14:49.534529 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:14:49.534529 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:14:49.534529 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:14:49.534529 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:14:49.534529 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:14:49.534529 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:14:49.534529 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:14:49.534529 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:14:49.534529 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:14:49.534529 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 02:14:49.534529 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 02:14:49.534529 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 02:14:49.534529 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 02:14:49.535516 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 02:14:49.535516 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 02:14:49.535516 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 02:14:49.535516 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 02:14:49.535516 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:14:49.535516 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:14:49.535516 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:14:49.535516 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:14:49.535516 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:14:49.535516 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:14:49.535516 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:14:49.535516 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:14:49.535516 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:14:49.535516 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:14:49.535516 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:14:49.535516 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:14:49.536325 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:14:49.536325 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:14:49.536325 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:14:49.536325 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:14:49.536325 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 02:14:49.536325 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 02:14:49.536325 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 02:14:49.536325 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 02:14:49.536325 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 02:14:49.536325 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 02:14:49.536325 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 02:14:49.536325 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:14:49.536325 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 134/260] 10.222.0.66 () {56 vars in 1114 bytes} [Mon Jul 18 02:14:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:14:49.544800 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcd66fa-3fc2-4350-a4d9-dce5ed86d954 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:49.609110 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:49.699784 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922e1630-5035-4256-9f3c-aa1269849c8f None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:49.761905 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcd66fa-3fc2-4350-a4d9-dce5ed86d954 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:49.762989 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0653e87-564e-4c64-a9cd-7e9d800b8b1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:49.763934 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0653e87-564e-4c64-a9cd-7e9d800b8b1b None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:49.764225 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0653e87-564e-4c64-a9cd-7e9d800b8b1b None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:49.764605 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0653e87-564e-4c64-a9cd-7e9d800b8b1b None admin] PATH_INFO: `/v3/users/046d41a707e04cce8aa98b4f4294da80` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:49.766484 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0653e87-564e-4c64-a9cd-7e9d800b8b1b None admin] RBAC: Authorizing `identity:get_user(user_id=046d41a707e04cce8aa98b4f4294da80)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:49.767907 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0653e87-564e-4c64-a9cd-7e9d800b8b1b None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:49.769713 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 127/261] 10.222.0.66 () {56 vars in 1177 bytes} [Mon Jul 18 02:14:49 2022] GET /identity/v3/users/046d41a707e04cce8aa98b4f4294da80 => generated 283 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:14:49.775264 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0653e87-564e-4c64-a9cd-7e9d800b8b1b None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:49.803646 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed4c068-9de0-40c6-9345-3a095b48323d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:49.804779 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed4c068-9de0-40c6-9345-3a095b48323d None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:49.805267 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed4c068-9de0-40c6-9345-3a095b48323d None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:49.805767 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed4c068-9de0-40c6-9345-3a095b48323d None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:49.806851 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed4c068-9de0-40c6-9345-3a095b48323d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:14:49.807453 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed4c068-9de0-40c6-9345-3a095b48323d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:49.809065 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed4c068-9de0-40c6-9345-3a095b48323d None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:49.819301 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 135/262] 10.222.0.66 () {56 vars in 1111 bytes} [Mon Jul 18 02:14:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:14:49.827118 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed4c068-9de0-40c6-9345-3a095b48323d None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:49.838680 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:49.886200 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0653e87-564e-4c64-a9cd-7e9d800b8b1b None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:49.914350 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:49.923969 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79daae81-8688-4f93-9cd1-0a942a91bcea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:49.924897 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79daae81-8688-4f93-9cd1-0a942a91bcea None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:49.925192 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79daae81-8688-4f93-9cd1-0a942a91bcea None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:49.925474 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79daae81-8688-4f93-9cd1-0a942a91bcea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:49.990358 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed4c068-9de0-40c6-9345-3a095b48323d None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:50.002199 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79daae81-8688-4f93-9cd1-0a942a91bcea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:50.002515 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79daae81-8688-4f93-9cd1-0a942a91bcea None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:50.047407 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47675af-2438-4dcd-b945-14e2910ebb92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:50.048265 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47675af-2438-4dcd-b945-14e2910ebb92 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:50.049052 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47675af-2438-4dcd-b945-14e2910ebb92 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:50.049320 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47675af-2438-4dcd-b945-14e2910ebb92 None admin] PATH_INFO: `/v3/users/046d41a707e04cce8aa98b4f4294da80` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:50.051002 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47675af-2438-4dcd-b945-14e2910ebb92 None admin] RBAC: Authorizing `identity:get_user(user_id=046d41a707e04cce8aa98b4f4294da80)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:50.052435 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47675af-2438-4dcd-b945-14e2910ebb92 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:50.054415 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 136/263] 10.222.0.66 () {56 vars in 1177 bytes} [Mon Jul 18 02:14:49 2022] GET /identity/v3/users/046d41a707e04cce8aa98b4f4294da80 => generated 283 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:14:50.059560 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47675af-2438-4dcd-b945-14e2910ebb92 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:50.114561 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 128/264] 10.222.0.66 () {58 vars in 1282 bytes} [Mon Jul 18 02:14:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 02:14:50.120801 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79daae81-8688-4f93-9cd1-0a942a91bcea None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:50.143848 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:50.150408 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:50.172110 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47675af-2438-4dcd-b945-14e2910ebb92 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:50.211610 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15567eca-1b7e-4772-bc22-029069de1f93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:50.212275 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15567eca-1b7e-4772-bc22-029069de1f93 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:50.212556 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15567eca-1b7e-4772-bc22-029069de1f93 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:50.212812 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15567eca-1b7e-4772-bc22-029069de1f93 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:50.231559 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79daae81-8688-4f93-9cd1-0a942a91bcea None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:50.268893 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5e377b-aacb-41c4-ad5d-177d5d7de07b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:50.270349 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5e377b-aacb-41c4-ad5d-177d5d7de07b None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:50.270930 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5e377b-aacb-41c4-ad5d-177d5d7de07b None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:50.271407 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5e377b-aacb-41c4-ad5d-177d5d7de07b None admin] PATH_INFO: `/v3/projects/9748772803a44ccea6827637294a3a97` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:50.272857 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5e377b-aacb-41c4-ad5d-177d5d7de07b None admin] RBAC: Authorizing `identity:get_project(project_id=9748772803a44ccea6827637294a3a97)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:50.274653 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5e377b-aacb-41c4-ad5d-177d5d7de07b None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:50.276906 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 129/265] 10.222.0.66 () {56 vars in 1186 bytes} [Mon Jul 18 02:14:50 2022] GET /identity/v3/projects/9748772803a44ccea6827637294a3a97 => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:14:50.286407 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5e377b-aacb-41c4-ad5d-177d5d7de07b None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:50.287394 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15567eca-1b7e-4772-bc22-029069de1f93 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:50.288887 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15567eca-1b7e-4772-bc22-029069de1f93 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:50.369808 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:50.399854 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 137/266] 10.222.0.66 () {58 vars in 1282 bytes} [Mon Jul 18 02:14:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 18 02:14:50.400718 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:50.407215 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15567eca-1b7e-4772-bc22-029069de1f93 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:50.409606 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5e377b-aacb-41c4-ad5d-177d5d7de07b None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:50.446562 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5dd024-49eb-4347-aab1-a063c90c1e73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:50.447643 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5dd024-49eb-4347-aab1-a063c90c1e73 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:50.448131 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5dd024-49eb-4347-aab1-a063c90c1e73 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:50.448635 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5dd024-49eb-4347-aab1-a063c90c1e73 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:50.449668 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5dd024-49eb-4347-aab1-a063c90c1e73 None admin] RBAC: Adding query filter params (role.id=d4aa3324f0c84887b41d0dc4acf7f30e, scope.project.id=9748772803a44ccea6827637294a3a97, user.id=046d41a707e04cce8aa98b4f4294da80) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:14:50.450274 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5dd024-49eb-4347-aab1-a063c90c1e73 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:50.451926 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5dd024-49eb-4347-aab1-a063c90c1e73 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:50.459140 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 130/267] 10.222.0.66 () {56 vars in 1374 bytes} [Mon Jul 18 02:14:50 2022] GET /identity/v3/role_assignments?user.id=046d41a707e04cce8aa98b4f4294da80&scope.project.id=9748772803a44ccea6827637294a3a97&role.id=d4aa3324f0c84887b41d0dc4acf7f30e => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 02:14:50.550099 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15567eca-1b7e-4772-bc22-029069de1f93 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:50.590001 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f4b746-d1d4-46f9-84a8-0897aaf93542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:50.590911 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f4b746-d1d4-46f9-84a8-0897aaf93542 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:50.591196 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f4b746-d1d4-46f9-84a8-0897aaf93542 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:50.591467 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f4b746-d1d4-46f9-84a8-0897aaf93542 None admin] PATH_INFO: `/v3/projects/9748772803a44ccea6827637294a3a97` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:50.592720 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f4b746-d1d4-46f9-84a8-0897aaf93542 None admin] RBAC: Authorizing `identity:get_project(project_id=9748772803a44ccea6827637294a3a97)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:50.594229 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f4b746-d1d4-46f9-84a8-0897aaf93542 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:50.597125 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 138/268] 10.222.0.66 () {56 vars in 1186 bytes} [Mon Jul 18 02:14:50 2022] GET /identity/v3/projects/9748772803a44ccea6827637294a3a97 => generated 302 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:14:50.602742 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5dd024-49eb-4347-aab1-a063c90c1e73 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:50.745914 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5dd024-49eb-4347-aab1-a063c90c1e73 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:50.783140 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1bee14-973d-4526-9a35-da8512c810a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:50.784202 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1bee14-973d-4526-9a35-da8512c810a8 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:50.784689 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1bee14-973d-4526-9a35-da8512c810a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:50.785166 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1bee14-973d-4526-9a35-da8512c810a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:50.786291 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1bee14-973d-4526-9a35-da8512c810a8 None admin] RBAC: Adding query filter params (role.id=a5b895323e7b4f0db2491abe9ad57ee9, scope.project.id=9748772803a44ccea6827637294a3a97, user.id=046d41a707e04cce8aa98b4f4294da80) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:14:50.786810 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1bee14-973d-4526-9a35-da8512c810a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:50.788486 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1bee14-973d-4526-9a35-da8512c810a8 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:50.795484 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 131/269] 10.222.0.66 () {56 vars in 1374 bytes} [Mon Jul 18 02:14:50 2022] GET /identity/v3/role_assignments?user.id=046d41a707e04cce8aa98b4f4294da80&scope.project.id=9748772803a44ccea6827637294a3a97&role.id=a5b895323e7b4f0db2491abe9ad57ee9 => generated 608 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:14:51.629778 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fae5cc-c10a-4cba-b77d-ce8e4df3c6be None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:51.630754 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fae5cc-c10a-4cba-b77d-ce8e4df3c6be None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:51.631230 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fae5cc-c10a-4cba-b77d-ce8e4df3c6be None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:51.633056 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 139/270] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:14:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:14:51.638726 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6695e9-16b4-4270-9907-deaacb247a9c None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:51.639568 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6695e9-16b4-4270-9907-deaacb247a9c None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:51.640054 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6695e9-16b4-4270-9907-deaacb247a9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:51.657578 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-3d6695e9-16b4-4270-9907-deaacb247a9c None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:14:51.659250 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-3d6695e9-16b4-4270-9907-deaacb247a9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:14:51.677323 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:51.731745 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7870306d-246e-4320-a3c1-49eed2d52d28 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:51.732301 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 132/271] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:14:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 02:14:51.732594 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7870306d-246e-4320-a3c1-49eed2d52d28 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:51.733066 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7870306d-246e-4320-a3c1-49eed2d52d28 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:51.734764 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 140/272] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:14:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 02:14:51.740262 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a090a62-89a4-437b-87db-b5a20d567fa7 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:51.740560 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a090a62-89a4-437b-87db-b5a20d567fa7 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:51.740842 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a090a62-89a4-437b-87db-b5a20d567fa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:51.758159 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-6a090a62-89a4-437b-87db-b5a20d567fa7 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:14:51.759298 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7870306d-246e-4320-a3c1-49eed2d52d28 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:51.759409 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-6a090a62-89a4-437b-87db-b5a20d567fa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:14:51.809209 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:51.831232 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 133/273] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:14:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 02:14:51.853998 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a00a57-7ace-473d-863c-d54ccb30c24a None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:51.854330 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a00a57-7ace-473d-863c-d54ccb30c24a None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:51.854842 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a00a57-7ace-473d-863c-d54ccb30c24a None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:51.856299 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 134/274] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:14:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 02:14:51.856978 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a00a57-7ace-473d-863c-d54ccb30c24a None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:51.872961 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7870306d-246e-4320-a3c1-49eed2d52d28 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:51.911039 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eedd67-e9d5-4734-9e26-aed082bb75d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:51.911833 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eedd67-e9d5-4734-9e26-aed082bb75d4 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:51.912117 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eedd67-e9d5-4734-9e26-aed082bb75d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:51.912389 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eedd67-e9d5-4734-9e26-aed082bb75d4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:51.924612 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9eedd67-e9d5-4734-9e26-aed082bb75d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 02:14:51.932962 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-d9eedd67-e9d5-4734-9e26-aed082bb75d4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 02:14:51.932962 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:14:51.932962 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:14:51.932962 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:14:51.932962 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:14:51.932962 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:14:51.932962 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:14:51.932962 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:14:51.932962 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:14:51.932962 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:14:51.932962 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:14:51.932962 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:14:51.932962 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 02:14:51.932962 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 02:14:51.932962 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 02:14:51.932962 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 02:14:51.933897 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 02:14:51.933897 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 02:14:51.933897 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 02:14:51.933897 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 02:14:51.933897 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:14:51.933897 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:14:51.933897 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:14:51.933897 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:14:51.933897 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:14:51.933897 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:14:51.933897 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:14:51.933897 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:14:51.933897 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:14:51.933897 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:14:51.933897 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:14:51.933897 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:14:51.934743 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:14:51.934743 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:14:51.934743 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:14:51.934743 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:14:51.934743 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 02:14:51.934743 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 02:14:51.934743 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 02:14:51.934743 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 02:14:51.934743 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 02:14:51.934743 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 02:14:51.934743 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 02:14:51.934743 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:14:51.939352 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 141/275] 10.222.0.66 () {56 vars in 1099 bytes} [Mon Jul 18 02:14:51 2022] GET /identity/v3/roles/reader => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:14:51.940557 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f606321f-d335-43e4-a6d4-003b04ee6c4a None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:51.940849 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f606321f-d335-43e4-a6d4-003b04ee6c4a None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:51.941132 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f606321f-d335-43e4-a6d4-003b04ee6c4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:51.943550 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:51.960882 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-f606321f-d335-43e4-a6d4-003b04ee6c4a None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:14:51.962074 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-f606321f-d335-43e4-a6d4-003b04ee6c4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:14:51.963108 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a00a57-7ace-473d-863c-d54ccb30c24a None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:52.000891 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b80941-db25-4619-945d-e52004f62009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:52.001968 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b80941-db25-4619-945d-e52004f62009 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:52.002292 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b80941-db25-4619-945d-e52004f62009 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:52.002567 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b80941-db25-4619-945d-e52004f62009 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:52.012826 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b80941-db25-4619-945d-e52004f62009 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 02:14:52.021204 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-89b80941-db25-4619-945d-e52004f62009 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 02:14:52.021204 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:14:52.021204 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:14:52.021204 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:14:52.021204 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:14:52.021204 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:14:52.021204 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:14:52.021204 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:14:52.021204 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:14:52.021204 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:14:52.021204 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:14:52.021204 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:14:52.021204 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 02:14:52.021204 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 02:14:52.021204 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 02:14:52.021204 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 02:14:52.022517 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 02:14:52.022517 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 02:14:52.022517 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 02:14:52.022517 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 02:14:52.022517 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:14:52.022517 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:14:52.022517 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:14:52.022517 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:14:52.022517 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:14:52.022517 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:14:52.022517 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:14:52.022517 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:14:52.022517 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:14:52.022517 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:14:52.022517 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:14:52.022517 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:14:52.023319 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:14:52.023319 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:14:52.023319 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:14:52.023319 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:14:52.023319 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 02:14:52.023319 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 02:14:52.023319 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 02:14:52.023319 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 02:14:52.023319 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 02:14:52.023319 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 02:14:52.023319 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 02:14:52.023319 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:14:52.023319 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 135/276] 10.222.0.66 () {56 vars in 1099 bytes} [Mon Jul 18 02:14:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:14:52.024019 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b80941-db25-4619-945d-e52004f62009 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:52.044308 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 142/277] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:14:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 02:14:52.045086 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f606321f-d335-43e4-a6d4-003b04ee6c4a None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:52.078423 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:52.137513 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b80941-db25-4619-945d-e52004f62009 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:52.150870 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f606321f-d335-43e4-a6d4-003b04ee6c4a None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:52.173943 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81993897-f363-4037-bc5f-6ee9cd79ef9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:52.175064 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81993897-f363-4037-bc5f-6ee9cd79ef9d None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:52.175350 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81993897-f363-4037-bc5f-6ee9cd79ef9d None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:52.175624 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81993897-f363-4037-bc5f-6ee9cd79ef9d None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:52.176381 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81993897-f363-4037-bc5f-6ee9cd79ef9d None admin] RBAC: Adding query filter params (name=reader) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:14:52.176689 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81993897-f363-4037-bc5f-6ee9cd79ef9d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:52.178075 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81993897-f363-4037-bc5f-6ee9cd79ef9d None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:52.186291 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 136/278] 10.222.0.66 () {56 vars in 1101 bytes} [Mon Jul 18 02:14:52 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:14:52.187069 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81993897-f363-4037-bc5f-6ee9cd79ef9d None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:52.191693 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f442f5b-350a-4219-aae4-cd3866c47191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:52.191991 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:52.192676 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f442f5b-350a-4219-aae4-cd3866c47191 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:52.192676 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f442f5b-350a-4219-aae4-cd3866c47191 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:52.192918 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f442f5b-350a-4219-aae4-cd3866c47191 None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:52.193643 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f442f5b-350a-4219-aae4-cd3866c47191 None admin] RBAC: Adding query filter params (name=member) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:14:52.193947 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f442f5b-350a-4219-aae4-cd3866c47191 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:52.195395 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f442f5b-350a-4219-aae4-cd3866c47191 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:52.203870 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 143/279] 10.222.0.66 () {56 vars in 1101 bytes} [Mon Jul 18 02:14:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:14:52.205435 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955a4592-8669-4792-8ce6-a309b8071bd8 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:52.205730 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955a4592-8669-4792-8ce6-a309b8071bd8 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:52.206006 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955a4592-8669-4792-8ce6-a309b8071bd8 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:52.207417 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 144/280] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:14:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:14:52.208039 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955a4592-8669-4792-8ce6-a309b8071bd8 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:52.288251 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81993897-f363-4037-bc5f-6ee9cd79ef9d None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:52.300959 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:52.308304 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955a4592-8669-4792-8ce6-a309b8071bd8 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:52.322553 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a19c85-f22f-49de-84c4-8338db040f9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:52.323344 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a19c85-f22f-49de-84c4-8338db040f9f None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:52.323624 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a19c85-f22f-49de-84c4-8338db040f9f None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:52.323898 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a19c85-f22f-49de-84c4-8338db040f9f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:52.325051 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a19c85-f22f-49de-84c4-8338db040f9f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:52.326529 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a19c85-f22f-49de-84c4-8338db040f9f None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:52.328158 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 137/281] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 18 02:14:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:14:52.329327 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf7c1bb-b5ce-4e4b-881e-41b599bf28a7 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:52.329684 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf7c1bb-b5ce-4e4b-881e-41b599bf28a7 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:52.329950 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf7c1bb-b5ce-4e4b-881e-41b599bf28a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:52.343296 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0990e1b6-8741-496f-b76e-c88445459be8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:52.344172 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0990e1b6-8741-496f-b76e-c88445459be8 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:52.346643 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0990e1b6-8741-496f-b76e-c88445459be8 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:52.346643 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0990e1b6-8741-496f-b76e-c88445459be8 None admin] PATH_INFO: `/v3/users/a6a181b5b3cf45b9b43ec3a81222b2c7` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:52.346811 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-5cf7c1bb-b5ce-4e4b-881e-41b599bf28a7 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:14:52.347264 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-5cf7c1bb-b5ce-4e4b-881e-41b599bf28a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:14:52.347361 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0990e1b6-8741-496f-b76e-c88445459be8 None admin] RBAC: Authorizing `identity:get_user(user_id=a6a181b5b3cf45b9b43ec3a81222b2c7)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:52.348739 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0990e1b6-8741-496f-b76e-c88445459be8 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:52.350419 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 145/282] 10.222.0.66 () {56 vars in 1177 bytes} [Mon Jul 18 02:14:52 2022] GET /identity/v3/users/a6a181b5b3cf45b9b43ec3a81222b2c7 => generated 297 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:14:52.351173 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0990e1b6-8741-496f-b76e-c88445459be8 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:52.413090 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 138/283] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:14:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 02:14:52.413902 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf7c1bb-b5ce-4e4b-881e-41b599bf28a7 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:52.416099 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:52.466921 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0990e1b6-8741-496f-b76e-c88445459be8 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:52.501147 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba8064a-f0ad-472c-b0a1-a70bc9962fa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:52.502012 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba8064a-f0ad-472c-b0a1-a70bc9962fa1 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:52.502318 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba8064a-f0ad-472c-b0a1-a70bc9962fa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:52.502711 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba8064a-f0ad-472c-b0a1-a70bc9962fa1 None admin] PATH_INFO: `/v3/users/96c2e6a1698743318d8807a127902a23` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:52.513347 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf7c1bb-b5ce-4e4b-881e-41b599bf28a7 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:52.515900 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba8064a-f0ad-472c-b0a1-a70bc9962fa1 None admin] RBAC: Authorizing `identity:get_user(user_id=96c2e6a1698743318d8807a127902a23)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:52.517723 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba8064a-f0ad-472c-b0a1-a70bc9962fa1 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:52.520096 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 146/284] 10.222.0.66 () {56 vars in 1177 bytes} [Mon Jul 18 02:14:52 2022] GET /identity/v3/users/96c2e6a1698743318d8807a127902a23 => generated 293 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:14:52.520773 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba8064a-f0ad-472c-b0a1-a70bc9962fa1 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:52.543528 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:52.548760 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbff5ce-d979-404c-b044-ae45117bf483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:52.549694 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbff5ce-d979-404c-b044-ae45117bf483 None admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:52.549986 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbff5ce-d979-404c-b044-ae45117bf483 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:52.550294 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbff5ce-d979-404c-b044-ae45117bf483 None admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:52.551005 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbff5ce-d979-404c-b044-ae45117bf483 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:52.552503 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbff5ce-d979-404c-b044-ae45117bf483 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:52.622040 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba8064a-f0ad-472c-b0a1-a70bc9962fa1 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:52.651750 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 139/285] 10.222.0.66 () {60 vars in 1132 bytes} [Mon Jul 18 02:14:52 2022] POST /identity/v3/users => generated 293 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:14:52.652578 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbff5ce-d979-404c-b044-ae45117bf483 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:52.662651 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcc4411-0b55-4606-bbf9-92f50567c551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:52.663631 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcc4411-0b55-4606-bbf9-92f50567c551 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:52.663861 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcc4411-0b55-4606-bbf9-92f50567c551 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:52.664137 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcc4411-0b55-4606-bbf9-92f50567c551 None admin] PATH_INFO: `/v3/projects/9748772803a44ccea6827637294a3a97` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:52.665434 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcc4411-0b55-4606-bbf9-92f50567c551 None admin] RBAC: Authorizing `identity:get_project(project_id=9748772803a44ccea6827637294a3a97)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:52.667264 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcc4411-0b55-4606-bbf9-92f50567c551 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:52.669289 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 147/286] 10.222.0.66 () {56 vars in 1186 bytes} [Mon Jul 18 02:14:52 2022] GET /identity/v3/projects/9748772803a44ccea6827637294a3a97 => generated 302 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:14:52.671176 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcc4411-0b55-4606-bbf9-92f50567c551 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:52.725818 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:52.774325 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:52.776825 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbff5ce-d979-404c-b044-ae45117bf483 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:52.789691 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcc4411-0b55-4606-bbf9-92f50567c551 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:52.818263 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f220a6b-4dc1-4b26-875b-91f1667f87e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:52.819350 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f220a6b-4dc1-4b26-875b-91f1667f87e0 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:52.819638 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f220a6b-4dc1-4b26-875b-91f1667f87e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:52.819914 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f220a6b-4dc1-4b26-875b-91f1667f87e0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:52.831017 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c276dcd-953c-4fae-960c-4e916ae3ae04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:52.832268 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c276dcd-953c-4fae-960c-4e916ae3ae04 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:52.832552 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c276dcd-953c-4fae-960c-4e916ae3ae04 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:52.832833 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c276dcd-953c-4fae-960c-4e916ae3ae04 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:52.833337 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f220a6b-4dc1-4b26-875b-91f1667f87e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 02:14:52.833704 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c276dcd-953c-4fae-960c-4e916ae3ae04 None admin] RBAC: Adding query filter params (role.id=65feabf3b27944a8b01a86c8ac608457, user.id=96c2e6a1698743318d8807a127902a23, scope.system=all) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:14:52.834015 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c276dcd-953c-4fae-960c-4e916ae3ae04 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:52.835502 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c276dcd-953c-4fae-960c-4e916ae3ae04 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:52.840901 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 148/287] 10.222.0.66 () {56 vars in 1308 bytes} [Mon Jul 18 02:14:52 2022] GET /identity/v3/role_assignments?user.id=96c2e6a1698743318d8807a127902a23&scope.system=all&role.id=65feabf3b27944a8b01a86c8ac608457 => generated 226 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:14:52.841131 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-6f220a6b-4dc1-4b26-875b-91f1667f87e0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 02:14:52.841131 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:14:52.841131 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:14:52.841131 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:14:52.841131 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:14:52.841131 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:14:52.841131 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:14:52.841131 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:14:52.841131 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:14:52.841131 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:14:52.841131 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:14:52.841131 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:14:52.841131 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 02:14:52.841131 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 02:14:52.841131 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 02:14:52.841131 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 02:14:52.842191 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 02:14:52.842191 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 02:14:52.842191 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 02:14:52.842191 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 02:14:52.842191 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:14:52.842191 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:14:52.842191 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:14:52.842191 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:14:52.842191 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:14:52.842191 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:14:52.842191 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:14:52.842191 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:14:52.842191 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:14:52.842191 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:14:52.842191 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:14:52.842191 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:14:52.843017 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:14:52.843017 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:14:52.843017 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:14:52.843017 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:14:52.843017 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 02:14:52.843017 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 02:14:52.843017 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 02:14:52.843017 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 02:14:52.843017 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 02:14:52.843017 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 02:14:52.843017 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 02:14:52.843017 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:14:52.844127 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c276dcd-953c-4fae-960c-4e916ae3ae04 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:52.844219 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 140/288] 10.222.0.66 () {56 vars in 1099 bytes} [Mon Jul 18 02:14:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:14:52.850233 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f220a6b-4dc1-4b26-875b-91f1667f87e0 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:52.975735 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c276dcd-953c-4fae-960c-4e916ae3ae04 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:52.982600 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f220a6b-4dc1-4b26-875b-91f1667f87e0 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:53.004353 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:53.030022 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a34eaf-5ed3-416a-80e7-67318adbe93c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:53.030788 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a34eaf-5ed3-416a-80e7-67318adbe93c None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:53.031402 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a34eaf-5ed3-416a-80e7-67318adbe93c None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:53.031402 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a34eaf-5ed3-416a-80e7-67318adbe93c None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:53.032112 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a34eaf-5ed3-416a-80e7-67318adbe93c None admin] RBAC: Adding query filter params (name=member) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:14:53.033120 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a34eaf-5ed3-416a-80e7-67318adbe93c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:53.034640 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a34eaf-5ed3-416a-80e7-67318adbe93c None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:53.036680 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baa38d6-6406-40dc-838a-a08bfc5b58f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:53.037575 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baa38d6-6406-40dc-838a-a08bfc5b58f5 None admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:53.037854 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baa38d6-6406-40dc-838a-a08bfc5b58f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:53.038154 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baa38d6-6406-40dc-838a-a08bfc5b58f5 None admin] PATH_INFO: `/v3/projects/9748772803a44ccea6827637294a3a97/users/a6a181b5b3cf45b9b43ec3a81222b2c7/roles/4045191c71164a3e9a009f23ab3b1406` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:53.040362 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baa38d6-6406-40dc-838a-a08bfc5b58f5 None admin] RBAC: Authorizing `identity:create_grant(project_id=9748772803a44ccea6827637294a3a97, user_id=a6a181b5b3cf45b9b43ec3a81222b2c7, role_id=4045191c71164a3e9a009f23ab3b1406)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:53.041810 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baa38d6-6406-40dc-838a-a08bfc5b58f5 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:53.051759 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 141/289] 10.222.0.66 () {56 vars in 1101 bytes} [Mon Jul 18 02:14:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:14:53.057381 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a34eaf-5ed3-416a-80e7-67318adbe93c None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:53.117785 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 149/290] 10.222.0.66 () {58 vars in 1439 bytes} [Mon Jul 18 02:14:52 2022] PUT /identity/v3/projects/9748772803a44ccea6827637294a3a97/users/a6a181b5b3cf45b9b43ec3a81222b2c7/roles/4045191c71164a3e9a009f23ab3b1406 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:14:53.176573 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a34eaf-5ed3-416a-80e7-67318adbe93c None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:53.214065 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0525d5f8-7df1-487f-8c86-16b3ad081c95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:53.214992 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0525d5f8-7df1-487f-8c86-16b3ad081c95 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:53.215262 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0525d5f8-7df1-487f-8c86-16b3ad081c95 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:53.215541 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0525d5f8-7df1-487f-8c86-16b3ad081c95 None admin] PATH_INFO: `/v3/users/fdd615f258a64e40b311274fb6639eb4` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:53.216782 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0525d5f8-7df1-487f-8c86-16b3ad081c95 None admin] RBAC: Authorizing `identity:get_user(user_id=fdd615f258a64e40b311274fb6639eb4)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:53.218374 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0525d5f8-7df1-487f-8c86-16b3ad081c95 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:53.221078 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 142/291] 10.222.0.66 () {56 vars in 1177 bytes} [Mon Jul 18 02:14:53 2022] GET /identity/v3/users/fdd615f258a64e40b311274fb6639eb4 => generated 297 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 02:14:53.226970 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baa38d6-6406-40dc-838a-a08bfc5b58f5 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:53.243643 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:53.353123 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baa38d6-6406-40dc-838a-a08bfc5b58f5 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:53.400154 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b97886c-72c9-45d6-b6f5-9b4ce5997534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:53.400826 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b97886c-72c9-45d6-b6f5-9b4ce5997534 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:53.401115 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b97886c-72c9-45d6-b6f5-9b4ce5997534 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:53.401394 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b97886c-72c9-45d6-b6f5-9b4ce5997534 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:53.477996 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b97886c-72c9-45d6-b6f5-9b4ce5997534 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:53.479419 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b97886c-72c9-45d6-b6f5-9b4ce5997534 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:53.498444 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:53.586378 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 150/292] 10.222.0.66 () {58 vars in 1282 bytes} [Mon Jul 18 02:14:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 02:14:53.595952 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0525d5f8-7df1-487f-8c86-16b3ad081c95 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:53.684581 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:53.745419 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0525d5f8-7df1-487f-8c86-16b3ad081c95 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:53.779381 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351342e2-ae92-4fb9-9135-6f8832637c42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:53.780569 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351342e2-ae92-4fb9-9135-6f8832637c42 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:53.781027 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351342e2-ae92-4fb9-9135-6f8832637c42 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:53.781471 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351342e2-ae92-4fb9-9135-6f8832637c42 None admin] PATH_INFO: `/v3/projects/9748772803a44ccea6827637294a3a97` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:53.782882 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351342e2-ae92-4fb9-9135-6f8832637c42 None admin] RBAC: Authorizing `identity:get_project(project_id=9748772803a44ccea6827637294a3a97)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:53.784506 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351342e2-ae92-4fb9-9135-6f8832637c42 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:53.786735 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 143/293] 10.222.0.66 () {56 vars in 1186 bytes} [Mon Jul 18 02:14:53 2022] GET /identity/v3/projects/9748772803a44ccea6827637294a3a97 => generated 302 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:14:53.794108 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b97886c-72c9-45d6-b6f5-9b4ce5997534 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:53.930120 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b97886c-72c9-45d6-b6f5-9b4ce5997534 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:53.958505 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:53.965487 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce449abe-7586-433e-a90c-efc3cf8c9357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:53.966179 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce449abe-7586-433e-a90c-efc3cf8c9357 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:53.966472 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce449abe-7586-433e-a90c-efc3cf8c9357 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:53.966751 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce449abe-7586-433e-a90c-efc3cf8c9357 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:53.967587 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce449abe-7586-433e-a90c-efc3cf8c9357 None admin] RBAC: Adding query filter params (role.id=65feabf3b27944a8b01a86c8ac608457, scope.project.id=9748772803a44ccea6827637294a3a97, user.id=fdd615f258a64e40b311274fb6639eb4) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:14:53.967900 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce449abe-7586-433e-a90c-efc3cf8c9357 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:53.969546 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce449abe-7586-433e-a90c-efc3cf8c9357 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:53.978103 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 151/294] 10.222.0.66 () {56 vars in 1374 bytes} [Mon Jul 18 02:14:53 2022] GET /identity/v3/role_assignments?user.id=fdd615f258a64e40b311274fb6639eb4&scope.project.id=9748772803a44ccea6827637294a3a97&role.id=65feabf3b27944a8b01a86c8ac608457 => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:14:55.013594 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eed89bf-7efd-42bd-8dbc-f09789b9d772 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:55.014692 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eed89bf-7efd-42bd-8dbc-f09789b9d772 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:55.015179 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eed89bf-7efd-42bd-8dbc-f09789b9d772 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:55.017574 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 144/295] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:14:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:14:55.023229 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aac0b3b-7e68-4998-bd81-6541d2a74aeb None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:55.024129 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aac0b3b-7e68-4998-bd81-6541d2a74aeb None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:55.024606 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aac0b3b-7e68-4998-bd81-6541d2a74aeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:55.042155 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-7aac0b3b-7e68-4998-bd81-6541d2a74aeb None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:14:55.043666 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-7aac0b3b-7e68-4998-bd81-6541d2a74aeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:14:55.115431 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 152/296] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:14:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 02:14:55.137014 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eed89bf-7efd-42bd-8dbc-f09789b9d772 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:55.171886 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4903770-28ac-4206-81c9-368ee01b298b None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:55.172186 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4903770-28ac-4206-81c9-368ee01b298b None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:55.172441 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4903770-28ac-4206-81c9-368ee01b298b None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:55.174945 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 153/297] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:14:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 02:14:55.179753 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3e9c43-8bd1-4695-8f5b-f72a43d0a4fa None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:55.180043 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3e9c43-8bd1-4695-8f5b-f72a43d0a4fa None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:55.180319 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3e9c43-8bd1-4695-8f5b-f72a43d0a4fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:55.197977 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-cb3e9c43-8bd1-4695-8f5b-f72a43d0a4fa None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:14:55.199186 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-cb3e9c43-8bd1-4695-8f5b-f72a43d0a4fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:14:55.261086 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eed89bf-7efd-42bd-8dbc-f09789b9d772 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:55.279802 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:55.284748 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 154/298] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:14:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 02:14:55.304353 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587ec870-47fa-4a28-8179-3c3ee003c417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:55.305433 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587ec870-47fa-4a28-8179-3c3ee003c417 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:55.305862 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587ec870-47fa-4a28-8179-3c3ee003c417 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:55.306354 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587ec870-47fa-4a28-8179-3c3ee003c417 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:55.308141 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3e9c43-8bd1-4695-8f5b-f72a43d0a4fa None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:55.322276 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587ec870-47fa-4a28-8179-3c3ee003c417 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 02:14:55.331089 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-587ec870-47fa-4a28-8179-3c3ee003c417 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 02:14:55.331089 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:14:55.331089 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:14:55.331089 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:14:55.331089 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:14:55.331089 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:14:55.331089 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:14:55.331089 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:14:55.331089 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:14:55.331089 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:14:55.331089 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:14:55.331089 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:14:55.331089 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 02:14:55.331089 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 02:14:55.331089 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 02:14:55.331089 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 02:14:55.332006 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 02:14:55.332006 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 02:14:55.332006 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 02:14:55.332006 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 02:14:55.332006 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:14:55.332006 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:14:55.332006 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:14:55.332006 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:14:55.332006 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:14:55.332006 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:14:55.332006 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:14:55.332006 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:14:55.332006 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:14:55.332006 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:14:55.332006 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:14:55.332006 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:14:55.333035 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:14:55.333035 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:14:55.333035 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:14:55.333035 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:14:55.333035 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 02:14:55.333035 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 02:14:55.333035 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 02:14:55.333035 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 02:14:55.333035 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 02:14:55.333035 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 02:14:55.333035 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 02:14:55.333035 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:14:55.333035 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 145/299] 10.222.0.66 () {56 vars in 1099 bytes} [Mon Jul 18 02:14:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:14:55.337940 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587ec870-47fa-4a28-8179-3c3ee003c417 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:55.353742 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:55.493337 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3e9c43-8bd1-4695-8f5b-f72a43d0a4fa None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:55.529052 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3039970-a206-45bb-a0d5-1a3232d56bd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:55.529908 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3039970-a206-45bb-a0d5-1a3232d56bd4 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:55.530127 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587ec870-47fa-4a28-8179-3c3ee003c417 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:55.530233 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3039970-a206-45bb-a0d5-1a3232d56bd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:55.530425 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3039970-a206-45bb-a0d5-1a3232d56bd4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:55.531591 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3039970-a206-45bb-a0d5-1a3232d56bd4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:55.533702 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3039970-a206-45bb-a0d5-1a3232d56bd4 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:55.539904 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 155/300] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 18 02:14:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 02:14:55.539904 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af4cec2-c577-4e20-8249-6f6428451762 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:55.539904 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af4cec2-c577-4e20-8249-6f6428451762 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:55.539904 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af4cec2-c577-4e20-8249-6f6428451762 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:55.539904 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 156/301] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:14:55 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:14:55.542925 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af4cec2-c577-4e20-8249-6f6428451762 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:55.570522 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a928d95-96f6-4400-ab3c-b56dbab089ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:55.571207 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a928d95-96f6-4400-ab3c-b56dbab089ae None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:55.571495 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a928d95-96f6-4400-ab3c-b56dbab089ae None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:55.571778 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a928d95-96f6-4400-ab3c-b56dbab089ae None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:55.572532 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a928d95-96f6-4400-ab3c-b56dbab089ae None admin] RBAC: Adding query filter params (name=member) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:14:55.572870 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a928d95-96f6-4400-ab3c-b56dbab089ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:55.574285 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a928d95-96f6-4400-ab3c-b56dbab089ae None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:55.583461 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 146/302] 10.222.0.66 () {56 vars in 1101 bytes} [Mon Jul 18 02:14:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:14:55.583461 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3459537e-7f35-4bf9-87d3-ee91ce9a15b6 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:55.583461 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3459537e-7f35-4bf9-87d3-ee91ce9a15b6 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:55.583461 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3459537e-7f35-4bf9-87d3-ee91ce9a15b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:55.604083 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-3459537e-7f35-4bf9-87d3-ee91ce9a15b6 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:14:55.605277 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-3459537e-7f35-4bf9-87d3-ee91ce9a15b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:14:55.640490 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:55.667461 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af4cec2-c577-4e20-8249-6f6428451762 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:55.670647 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:55.689179 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 147/303] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:14:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 02:14:55.689179 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3459537e-7f35-4bf9-87d3-ee91ce9a15b6 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:55.717665 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5409729-6611-4c46-a94e-e46277088d14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:55.718370 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5409729-6611-4c46-a94e-e46277088d14 None admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:55.718654 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5409729-6611-4c46-a94e-e46277088d14 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:55.718932 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5409729-6611-4c46-a94e-e46277088d14 None admin] PATH_INFO: `/v3/groups` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:55.720251 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5409729-6611-4c46-a94e-e46277088d14 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:55.721707 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5409729-6611-4c46-a94e-e46277088d14 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:55.784081 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3459537e-7f35-4bf9-87d3-ee91ce9a15b6 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:55.798121 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 157/304] 10.222.0.66 () {60 vars in 1134 bytes} [Mon Jul 18 02:14:55 2022] POST /identity/v3/groups => generated 229 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:14:55.799017 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5409729-6611-4c46-a94e-e46277088d14 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:55.821139 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da8551d-65e3-45f3-94bc-3a57ca36945b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:55.822044 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da8551d-65e3-45f3-94bc-3a57ca36945b None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:55.822364 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da8551d-65e3-45f3-94bc-3a57ca36945b None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:55.822646 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da8551d-65e3-45f3-94bc-3a57ca36945b None admin] PATH_INFO: `/v3/users/96c2e6a1698743318d8807a127902a23` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:55.823957 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da8551d-65e3-45f3-94bc-3a57ca36945b None admin] RBAC: Authorizing `identity:get_user(user_id=96c2e6a1698743318d8807a127902a23)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:55.825471 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da8551d-65e3-45f3-94bc-3a57ca36945b None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:55.827477 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 148/305] 10.222.0.66 () {56 vars in 1177 bytes} [Mon Jul 18 02:14:55 2022] GET /identity/v3/users/96c2e6a1698743318d8807a127902a23 => generated 293 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:14:55.828554 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d533981-6bd8-411c-9017-505ecf7f83ae None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:55.829375 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d533981-6bd8-411c-9017-505ecf7f83ae None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:55.829854 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d533981-6bd8-411c-9017-505ecf7f83ae None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:55.831530 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 149/306] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:14:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:14:55.835462 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d533981-6bd8-411c-9017-505ecf7f83ae None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:55.880091 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:55.924599 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5409729-6611-4c46-a94e-e46277088d14 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:55.958027 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d533981-6bd8-411c-9017-505ecf7f83ae None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:55.964578 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d573ae-4256-4dbc-a6a2-0275b0f30345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:55.965494 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d573ae-4256-4dbc-a6a2-0275b0f30345 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:55.966484 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d573ae-4256-4dbc-a6a2-0275b0f30345 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:55.966669 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d573ae-4256-4dbc-a6a2-0275b0f30345 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:55.982229 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d573ae-4256-4dbc-a6a2-0275b0f30345 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 02:14:55.990963 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-49d573ae-4256-4dbc-a6a2-0275b0f30345 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 02:14:55.990963 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:14:55.990963 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:14:55.990963 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:14:55.990963 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:14:55.990963 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:14:55.990963 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:14:55.990963 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:14:55.990963 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:14:55.990963 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:14:55.990963 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:14:55.990963 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:14:55.990963 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 02:14:55.990963 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 02:14:55.990963 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 02:14:55.990963 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 02:14:55.991907 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 02:14:55.991907 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 02:14:55.991907 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 02:14:55.991907 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 02:14:55.991907 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:14:55.991907 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:14:55.991907 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:14:55.991907 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:14:55.991907 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:14:55.991907 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:14:55.991907 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:14:55.991907 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:14:55.991907 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:14:55.991907 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:14:55.991907 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:14:55.991907 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:14:55.992728 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:14:55.992728 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:14:55.992728 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:14:55.992728 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:14:55.992728 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 02:14:55.992728 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 02:14:55.992728 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 02:14:55.992728 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 02:14:55.992728 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 02:14:55.992728 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 02:14:55.992728 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 02:14:55.992728 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:14:55.992728 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 158/307] 10.222.0.66 () {56 vars in 1099 bytes} [Mon Jul 18 02:14:55 2022] GET /identity/v3/roles/reader => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:14:55.995325 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512c19fa-ee57-49bf-ac56-c63c3856a500 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:55.995615 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512c19fa-ee57-49bf-ac56-c63c3856a500 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:55.995896 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512c19fa-ee57-49bf-ac56-c63c3856a500 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:55.996422 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096f0c1b-5330-4f7c-b913-3a2666d1b5c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:55.997299 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096f0c1b-5330-4f7c-b913-3a2666d1b5c5 None admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:55.997582 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096f0c1b-5330-4f7c-b913-3a2666d1b5c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:55.997923 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096f0c1b-5330-4f7c-b913-3a2666d1b5c5 None admin] PATH_INFO: `/v3/system/users/96c2e6a1698743318d8807a127902a23/roles/65feabf3b27944a8b01a86c8ac608457` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:55.999389 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096f0c1b-5330-4f7c-b913-3a2666d1b5c5 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=96c2e6a1698743318d8807a127902a23, role_id=65feabf3b27944a8b01a86c8ac608457)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:56.000876 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096f0c1b-5330-4f7c-b913-3a2666d1b5c5 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:56.013693 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-512c19fa-ee57-49bf-ac56-c63c3856a500 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:14:56.014981 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-512c19fa-ee57-49bf-ac56-c63c3856a500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:14:56.033216 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:56.081222 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 159/308] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:14:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 02:14:56.081809 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512c19fa-ee57-49bf-ac56-c63c3856a500 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:56.120481 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 150/309] 10.222.0.66 () {58 vars in 1334 bytes} [Mon Jul 18 02:14:55 2022] PUT /identity/v3/system/users/96c2e6a1698743318d8807a127902a23/roles/65feabf3b27944a8b01a86c8ac608457 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:14:56.121247 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096f0c1b-5330-4f7c-b913-3a2666d1b5c5 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:56.205704 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512c19fa-ee57-49bf-ac56-c63c3856a500 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:56.248965 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a2e7ae-599e-45d0-a0a0-a1bb1aa794a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:56.249992 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a2e7ae-599e-45d0-a0a0-a1bb1aa794a5 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:56.250515 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a2e7ae-599e-45d0-a0a0-a1bb1aa794a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:56.251942 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a2e7ae-599e-45d0-a0a0-a1bb1aa794a5 None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:56.253263 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a2e7ae-599e-45d0-a0a0-a1bb1aa794a5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:14:56.253885 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a2e7ae-599e-45d0-a0a0-a1bb1aa794a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:56.255568 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a2e7ae-599e-45d0-a0a0-a1bb1aa794a5 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:56.260713 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:56.264887 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096f0c1b-5330-4f7c-b913-3a2666d1b5c5 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:56.272695 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 160/310] 10.222.0.66 () {56 vars in 1101 bytes} [Mon Jul 18 02:14:56 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:14:56.274337 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a2e7ae-599e-45d0-a0a0-a1bb1aa794a5 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:56.297453 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:56.306765 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ad8936-f147-4e2e-94ce-fb3a92a93d6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:56.307825 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ad8936-f147-4e2e-94ce-fb3a92a93d6c None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:56.308263 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ad8936-f147-4e2e-94ce-fb3a92a93d6c None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:56.308545 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ad8936-f147-4e2e-94ce-fb3a92a93d6c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:56.319700 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ad8936-f147-4e2e-94ce-fb3a92a93d6c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 02:14:56.327620 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-b6ad8936-f147-4e2e-94ce-fb3a92a93d6c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 02:14:56.327620 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:14:56.327620 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:14:56.327620 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:14:56.327620 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:14:56.327620 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:14:56.327620 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:14:56.327620 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:14:56.327620 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:14:56.327620 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:14:56.327620 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:14:56.327620 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:14:56.327620 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 02:14:56.327620 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 02:14:56.327620 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 02:14:56.327620 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 02:14:56.328558 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 02:14:56.328558 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 02:14:56.328558 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 02:14:56.328558 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 02:14:56.328558 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:14:56.328558 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:14:56.328558 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:14:56.328558 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:14:56.328558 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:14:56.328558 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:14:56.328558 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:14:56.328558 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:14:56.328558 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:14:56.328558 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:14:56.328558 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:14:56.328558 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:14:56.329968 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:14:56.329968 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:14:56.329968 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:14:56.329968 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:14:56.329968 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 02:14:56.329968 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 02:14:56.329968 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 02:14:56.329968 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 02:14:56.329968 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 02:14:56.329968 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 02:14:56.329968 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 02:14:56.329968 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:14:56.329968 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 151/311] 10.222.0.66 () {56 vars in 1099 bytes} [Mon Jul 18 02:14:56 2022] GET /identity/v3/roles/reader => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:14:56.336356 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ad8936-f147-4e2e-94ce-fb3a92a93d6c None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:56.390614 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a2e7ae-599e-45d0-a0a0-a1bb1aa794a5 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:56.432471 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9e9897-da22-48df-9888-4d62734cb780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:56.433633 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9e9897-da22-48df-9888-4d62734cb780 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:56.433916 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9e9897-da22-48df-9888-4d62734cb780 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:56.434220 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9e9897-da22-48df-9888-4d62734cb780 None admin] PATH_INFO: `/v3/users/140d8b9d73914a76b5bb6f278cfe4098` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:56.449630 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9e9897-da22-48df-9888-4d62734cb780 None admin] RBAC: Authorizing `identity:get_user(user_id=140d8b9d73914a76b5bb6f278cfe4098)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:56.451143 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9e9897-da22-48df-9888-4d62734cb780 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:56.451861 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ad8936-f147-4e2e-94ce-fb3a92a93d6c None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:56.453322 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 161/312] 10.222.0.66 () {56 vars in 1177 bytes} [Mon Jul 18 02:14:56 2022] GET /identity/v3/users/140d8b9d73914a76b5bb6f278cfe4098 => generated 293 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 02:14:56.460087 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9e9897-da22-48df-9888-4d62734cb780 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:56.500097 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321b70ad-cac2-411d-a590-ed46ba8f48ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:56.500874 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321b70ad-cac2-411d-a590-ed46ba8f48ee None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:56.501160 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321b70ad-cac2-411d-a590-ed46ba8f48ee None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:56.501441 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321b70ad-cac2-411d-a590-ed46ba8f48ee None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:56.502396 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321b70ad-cac2-411d-a590-ed46ba8f48ee None admin] RBAC: Adding query filter params (name=reader) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:14:56.502707 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321b70ad-cac2-411d-a590-ed46ba8f48ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:56.505056 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321b70ad-cac2-411d-a590-ed46ba8f48ee None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:56.515512 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 152/313] 10.222.0.66 () {56 vars in 1101 bytes} [Mon Jul 18 02:14:56 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 02:14:56.521305 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321b70ad-cac2-411d-a590-ed46ba8f48ee None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:56.552111 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:56.555127 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:56.583783 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9e9897-da22-48df-9888-4d62734cb780 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:56.643595 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5084585f-be7a-4d7e-9de8-112d7018b4c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:56.644738 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5084585f-be7a-4d7e-9de8-112d7018b4c4 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:56.645292 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5084585f-be7a-4d7e-9de8-112d7018b4c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:56.645759 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5084585f-be7a-4d7e-9de8-112d7018b4c4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:56.646810 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5084585f-be7a-4d7e-9de8-112d7018b4c4 None admin] RBAC: Adding query filter params (role.id=4045191c71164a3e9a009f23ab3b1406, user.id=140d8b9d73914a76b5bb6f278cfe4098, scope.system=all) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:14:56.647363 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5084585f-be7a-4d7e-9de8-112d7018b4c4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:56.649110 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5084585f-be7a-4d7e-9de8-112d7018b4c4 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:56.650690 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321b70ad-cac2-411d-a590-ed46ba8f48ee None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:56.655194 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 162/314] 10.222.0.66 () {56 vars in 1308 bytes} [Mon Jul 18 02:14:56 2022] GET /identity/v3/role_assignments?user.id=140d8b9d73914a76b5bb6f278cfe4098&scope.system=all&role.id=4045191c71164a3e9a009f23ab3b1406 => generated 226 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:14:56.691189 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03520cf8-2098-4fc7-aafa-9dd901a0445f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:56.692090 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03520cf8-2098-4fc7-aafa-9dd901a0445f None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:56.692372 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03520cf8-2098-4fc7-aafa-9dd901a0445f None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:56.692650 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03520cf8-2098-4fc7-aafa-9dd901a0445f None admin] PATH_INFO: `/v3/users/a6a181b5b3cf45b9b43ec3a81222b2c7` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:56.693943 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03520cf8-2098-4fc7-aafa-9dd901a0445f None admin] RBAC: Authorizing `identity:get_user(user_id=a6a181b5b3cf45b9b43ec3a81222b2c7)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:56.695512 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03520cf8-2098-4fc7-aafa-9dd901a0445f None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:56.697528 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 153/315] 10.222.0.66 () {56 vars in 1177 bytes} [Mon Jul 18 02:14:56 2022] GET /identity/v3/users/a6a181b5b3cf45b9b43ec3a81222b2c7 => generated 297 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 02:14:56.702950 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5084585f-be7a-4d7e-9de8-112d7018b4c4 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:56.819017 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5084585f-be7a-4d7e-9de8-112d7018b4c4 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:56.857799 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8849c4c-a489-4966-b6db-9a083e260432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:56.858839 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8849c4c-a489-4966-b6db-9a083e260432 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:56.859319 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8849c4c-a489-4966-b6db-9a083e260432 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:56.859819 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8849c4c-a489-4966-b6db-9a083e260432 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:56.869454 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:56.930762 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8849c4c-a489-4966-b6db-9a083e260432 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:56.932558 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8849c4c-a489-4966-b6db-9a083e260432 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:57.041705 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 163/316] 10.222.0.66 () {58 vars in 1282 bytes} [Mon Jul 18 02:14:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 02:14:57.046920 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03520cf8-2098-4fc7-aafa-9dd901a0445f None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:57.141109 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:57.153186 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03520cf8-2098-4fc7-aafa-9dd901a0445f None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:57.188752 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea89a6e-4366-41d7-bcf2-bacd671f163e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:57.190314 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea89a6e-4366-41d7-bcf2-bacd671f163e None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:57.190599 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea89a6e-4366-41d7-bcf2-bacd671f163e None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:57.190876 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea89a6e-4366-41d7-bcf2-bacd671f163e None admin] PATH_INFO: `/v3/projects/9748772803a44ccea6827637294a3a97` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:57.192082 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea89a6e-4366-41d7-bcf2-bacd671f163e None admin] RBAC: Authorizing `identity:get_project(project_id=9748772803a44ccea6827637294a3a97)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:57.193556 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea89a6e-4366-41d7-bcf2-bacd671f163e None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:57.195614 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 154/317] 10.222.0.66 () {56 vars in 1186 bytes} [Mon Jul 18 02:14:57 2022] GET /identity/v3/projects/9748772803a44ccea6827637294a3a97 => generated 302 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:14:57.201381 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8849c4c-a489-4966-b6db-9a083e260432 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:57.242561 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:57.303232 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8849c4c-a489-4966-b6db-9a083e260432 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:57.342814 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bf9d90-8451-4765-a319-a3ac266a1f83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:57.343485 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bf9d90-8451-4765-a319-a3ac266a1f83 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:57.343783 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bf9d90-8451-4765-a319-a3ac266a1f83 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:57.344063 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bf9d90-8451-4765-a319-a3ac266a1f83 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:57.344939 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bf9d90-8451-4765-a319-a3ac266a1f83 None admin] RBAC: Adding query filter params (role.id=4045191c71164a3e9a009f23ab3b1406, scope.project.id=9748772803a44ccea6827637294a3a97, user.id=a6a181b5b3cf45b9b43ec3a81222b2c7) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:14:57.345253 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bf9d90-8451-4765-a319-a3ac266a1f83 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:57.346745 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bf9d90-8451-4765-a319-a3ac266a1f83 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:57.353111 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 164/318] 10.222.0.66 () {56 vars in 1374 bytes} [Mon Jul 18 02:14:57 2022] GET /identity/v3/role_assignments?user.id=a6a181b5b3cf45b9b43ec3a81222b2c7&scope.project.id=9748772803a44ccea6827637294a3a97&role.id=4045191c71164a3e9a009f23ab3b1406 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:14:57.993688 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96c8b5e-c66f-4502-afa5-088a49db280d None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:57.994127 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96c8b5e-c66f-4502-afa5-088a49db280d None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:57.994309 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96c8b5e-c66f-4502-afa5-088a49db280d None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:57.995822 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 155/319] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:14:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:14:58.000991 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1850b4e-c146-4a9a-851a-2e06235fc63d None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:58.001287 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1850b4e-c146-4a9a-851a-2e06235fc63d None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:58.001571 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1850b4e-c146-4a9a-851a-2e06235fc63d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:58.019491 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-d1850b4e-c146-4a9a-851a-2e06235fc63d None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:14:58.020683 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-d1850b4e-c146-4a9a-851a-2e06235fc63d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:14:58.089549 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 165/320] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:14:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 02:14:58.112820 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96c8b5e-c66f-4502-afa5-088a49db280d None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:58.217586 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96c8b5e-c66f-4502-afa5-088a49db280d None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:58.243160 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2c4b40-88a4-4a5a-902c-686f40351a8d None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:58.243485 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2c4b40-88a4-4a5a-902c-686f40351a8d None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:58.243815 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2c4b40-88a4-4a5a-902c-686f40351a8d None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:58.245469 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 166/321] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:14:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 02:14:58.251332 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75d6d4b-547e-4930-a8e1-8c34411fe99b None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:58.251622 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75d6d4b-547e-4930-a8e1-8c34411fe99b None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:58.251901 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75d6d4b-547e-4930-a8e1-8c34411fe99b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:58.254437 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b44e06-6d78-470f-a6ca-12031e2bbf07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:58.255238 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b44e06-6d78-470f-a6ca-12031e2bbf07 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:58.255519 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b44e06-6d78-470f-a6ca-12031e2bbf07 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:58.255790 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b44e06-6d78-470f-a6ca-12031e2bbf07 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:58.256941 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b44e06-6d78-470f-a6ca-12031e2bbf07 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:58.258421 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b44e06-6d78-470f-a6ca-12031e2bbf07 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:58.264579 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 156/322] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 18 02:14:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:14:58.266276 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b44e06-6d78-470f-a6ca-12031e2bbf07 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:58.270973 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-f75d6d4b-547e-4930-a8e1-8c34411fe99b None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:14:58.272145 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-f75d6d4b-547e-4930-a8e1-8c34411fe99b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:14:58.294459 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:58.300703 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:58.342397 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 167/323] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:14:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 02:14:58.366733 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75d6d4b-547e-4930-a8e1-8c34411fe99b None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:58.376076 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b44e06-6d78-470f-a6ca-12031e2bbf07 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:58.474472 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6bdccd-2357-408c-ba40-ee948087cee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:58.475148 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6bdccd-2357-408c-ba40-ee948087cee5 None admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:58.475434 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6bdccd-2357-408c-ba40-ee948087cee5 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:58.475713 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6bdccd-2357-408c-ba40-ee948087cee5 None admin] PATH_INFO: `/v3/groups` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:58.476407 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6bdccd-2357-408c-ba40-ee948087cee5 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:58.477863 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6bdccd-2357-408c-ba40-ee948087cee5 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:58.538908 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75d6d4b-547e-4930-a8e1-8c34411fe99b None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:58.573107 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab6d199-9899-4eb3-8ca0-97ef09d374ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:58.573894 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab6d199-9899-4eb3-8ca0-97ef09d374ce None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:58.574202 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab6d199-9899-4eb3-8ca0-97ef09d374ce None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:58.574479 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab6d199-9899-4eb3-8ca0-97ef09d374ce None admin] PATH_INFO: `/v3/roles/member` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:58.585461 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab6d199-9899-4eb3-8ca0-97ef09d374ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 02:14:58.593359 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-6ab6d199-9899-4eb3-8ca0-97ef09d374ce None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 02:14:58.593359 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:14:58.593359 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:14:58.593359 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:14:58.593359 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:14:58.593359 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:14:58.593359 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:14:58.593359 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:14:58.593359 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:14:58.593359 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:14:58.593359 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:14:58.593359 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:14:58.593359 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 02:14:58.593359 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 02:14:58.593359 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 02:14:58.593359 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 02:14:58.594516 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 02:14:58.594516 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 02:14:58.594516 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 02:14:58.594516 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 02:14:58.594516 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:14:58.594516 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:14:58.594516 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:14:58.594516 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:14:58.594516 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:14:58.594516 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:14:58.594516 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:14:58.594516 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:14:58.594516 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:14:58.594516 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:14:58.594516 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:14:58.594516 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:14:58.595705 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:14:58.595705 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:14:58.595705 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:14:58.595705 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:14:58.595705 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 02:14:58.595705 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 02:14:58.595705 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 02:14:58.595705 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 02:14:58.595705 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 02:14:58.595705 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 02:14:58.595705 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 02:14:58.595705 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:14:58.595705 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 168/324] 10.222.0.66 () {56 vars in 1099 bytes} [Mon Jul 18 02:14:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 02:14:58.601696 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab6d199-9899-4eb3-8ca0-97ef09d374ce None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:58.626789 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:58.635002 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 157/325] 10.222.0.66 () {60 vars in 1134 bytes} [Mon Jul 18 02:14:58 2022] POST /identity/v3/groups => generated 226 bytes in 369 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 02:14:58.710043 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab6d199-9899-4eb3-8ca0-97ef09d374ce None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:58.726905 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1956de59-026b-4ff0-b43e-660e272835ae None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:58.727192 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1956de59-026b-4ff0-b43e-660e272835ae None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:58.727472 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1956de59-026b-4ff0-b43e-660e272835ae None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:58.729086 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 158/326] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:14:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 02:14:58.734405 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2b3460-20a5-4f96-8583-06c0232f190e None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:58.734650 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2b3460-20a5-4f96-8583-06c0232f190e None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:58.735481 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2b3460-20a5-4f96-8583-06c0232f190e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:58.748443 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b89358-9599-4b32-9ab7-836d541029cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:58.749128 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b89358-9599-4b32-9ab7-836d541029cc None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:58.749468 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b89358-9599-4b32-9ab7-836d541029cc None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:58.749749 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b89358-9599-4b32-9ab7-836d541029cc None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:58.751298 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b89358-9599-4b32-9ab7-836d541029cc None admin] RBAC: Adding query filter params (name=member) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:14:58.751654 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b89358-9599-4b32-9ab7-836d541029cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:58.753098 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b89358-9599-4b32-9ab7-836d541029cc None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:58.753592 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-2d2b3460-20a5-4f96-8583-06c0232f190e None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:14:58.754948 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-2d2b3460-20a5-4f96-8583-06c0232f190e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:14:58.767441 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 169/327] 10.222.0.66 () {56 vars in 1101 bytes} [Mon Jul 18 02:14:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 02:14:58.768692 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b89358-9599-4b32-9ab7-836d541029cc None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:58.849623 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 159/328] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:14:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 02:14:58.863021 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:58.871723 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2b3460-20a5-4f96-8583-06c0232f190e None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:58.884930 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b89358-9599-4b32-9ab7-836d541029cc None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:58.896463 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:58.925824 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca470fce-cb36-4912-a99f-5a247e5d4bb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:58.926737 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca470fce-cb36-4912-a99f-5a247e5d4bb6 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:58.927029 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca470fce-cb36-4912-a99f-5a247e5d4bb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:58.927303 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca470fce-cb36-4912-a99f-5a247e5d4bb6 None admin] PATH_INFO: `/v3/users/96c2e6a1698743318d8807a127902a23` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:58.928531 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca470fce-cb36-4912-a99f-5a247e5d4bb6 None admin] RBAC: Authorizing `identity:get_user(user_id=96c2e6a1698743318d8807a127902a23)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:58.930040 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca470fce-cb36-4912-a99f-5a247e5d4bb6 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:58.931937 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 170/329] 10.222.0.66 () {56 vars in 1177 bytes} [Mon Jul 18 02:14:58 2022] GET /identity/v3/users/96c2e6a1698743318d8807a127902a23 => generated 293 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:14:58.937460 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca470fce-cb36-4912-a99f-5a247e5d4bb6 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:59.007852 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2b3460-20a5-4f96-8583-06c0232f190e None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:59.055165 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eae2694-85be-44b3-a3a8-7ff8c2920bb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:59.056121 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eae2694-85be-44b3-a3a8-7ff8c2920bb7 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:59.056844 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eae2694-85be-44b3-a3a8-7ff8c2920bb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:59.057132 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eae2694-85be-44b3-a3a8-7ff8c2920bb7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:59.070578 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eae2694-85be-44b3-a3a8-7ff8c2920bb7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 02:14:59.081002 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-1eae2694-85be-44b3-a3a8-7ff8c2920bb7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 02:14:59.081002 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:14:59.081002 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:14:59.081002 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:14:59.081002 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:14:59.081002 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:14:59.081002 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:14:59.081002 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:14:59.081002 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:14:59.081002 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:14:59.081002 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:14:59.081002 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:14:59.081002 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 02:14:59.081002 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 02:14:59.081002 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 02:14:59.081002 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 02:14:59.082250 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 02:14:59.082250 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 02:14:59.082250 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 02:14:59.082250 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 02:14:59.082250 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:14:59.082250 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:14:59.082250 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:14:59.082250 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:14:59.082250 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:14:59.082250 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:14:59.082250 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:14:59.082250 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:14:59.082250 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:14:59.082250 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:14:59.082250 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:14:59.082250 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:14:59.083382 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:14:59.083382 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:14:59.083382 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:14:59.083382 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:14:59.083382 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 02:14:59.083382 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 02:14:59.083382 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 02:14:59.083382 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 02:14:59.083382 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 02:14:59.083382 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 02:14:59.083382 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 02:14:59.083382 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:14:59.083382 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 160/330] 10.222.0.66 () {56 vars in 1099 bytes} [Mon Jul 18 02:14:58 2022] GET /identity/v3/roles/reader => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 02:14:59.088278 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eae2694-85be-44b3-a3a8-7ff8c2920bb7 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:59.090961 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca470fce-cb36-4912-a99f-5a247e5d4bb6 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:59.136093 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc945ec-46e9-4799-9b7f-32977dd3d632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:59.136713 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc945ec-46e9-4799-9b7f-32977dd3d632 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:59.136997 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc945ec-46e9-4799-9b7f-32977dd3d632 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:59.137842 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc945ec-46e9-4799-9b7f-32977dd3d632 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:59.138739 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc945ec-46e9-4799-9b7f-32977dd3d632 None admin] RBAC: Adding query filter params (role.id=65feabf3b27944a8b01a86c8ac608457, user.id=96c2e6a1698743318d8807a127902a23, scope.system=all) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:14:59.139055 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc945ec-46e9-4799-9b7f-32977dd3d632 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:59.140981 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc945ec-46e9-4799-9b7f-32977dd3d632 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:59.157843 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 171/331] 10.222.0.66 () {56 vars in 1308 bytes} [Mon Jul 18 02:14:58 2022] GET /identity/v3/role_assignments?user.id=96c2e6a1698743318d8807a127902a23&scope.system=all&role.id=65feabf3b27944a8b01a86c8ac608457 => generated 510 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 02:14:59.188523 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:59.213586 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eae2694-85be-44b3-a3a8-7ff8c2920bb7 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:59.254633 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade728e1-67b6-4278-af42-9fdb96364e89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:59.255417 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade728e1-67b6-4278-af42-9fdb96364e89 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:59.255603 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade728e1-67b6-4278-af42-9fdb96364e89 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:59.255882 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade728e1-67b6-4278-af42-9fdb96364e89 None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:59.256654 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade728e1-67b6-4278-af42-9fdb96364e89 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:14:59.256960 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade728e1-67b6-4278-af42-9fdb96364e89 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:59.258424 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade728e1-67b6-4278-af42-9fdb96364e89 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:59.268758 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 161/332] 10.222.0.66 () {56 vars in 1101 bytes} [Mon Jul 18 02:14:59 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:14:59.274539 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc945ec-46e9-4799-9b7f-32977dd3d632 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:59.289741 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:14:59.389541 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc945ec-46e9-4799-9b7f-32977dd3d632 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:59.426855 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9aced0a-fb0c-454c-87bf-c1acce1f53c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:59.428126 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aced0a-fb0c-454c-87bf-c1acce1f53c7 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:59.428613 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aced0a-fb0c-454c-87bf-c1acce1f53c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:59.429083 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aced0a-fb0c-454c-87bf-c1acce1f53c7 None admin] PATH_INFO: `/v3/users/140d8b9d73914a76b5bb6f278cfe4098` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:59.430690 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9aced0a-fb0c-454c-87bf-c1acce1f53c7 None admin] RBAC: Authorizing `identity:get_user(user_id=140d8b9d73914a76b5bb6f278cfe4098)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:59.432466 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9aced0a-fb0c-454c-87bf-c1acce1f53c7 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:59.434697 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 172/333] 10.222.0.66 () {56 vars in 1177 bytes} [Mon Jul 18 02:14:59 2022] GET /identity/v3/users/140d8b9d73914a76b5bb6f278cfe4098 => generated 293 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:14:59.440150 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade728e1-67b6-4278-af42-9fdb96364e89 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:14:59.571677 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade728e1-67b6-4278-af42-9fdb96364e89 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:14:59.616227 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43aeaacc-9153-4a60-9011-d6f56e0deb54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:14:59.617102 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aeaacc-9153-4a60-9011-d6f56e0deb54 None admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:14:59.617381 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aeaacc-9153-4a60-9011-d6f56e0deb54 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:14:59.617659 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aeaacc-9153-4a60-9011-d6f56e0deb54 None admin] PATH_INFO: `/v3/system/users/140d8b9d73914a76b5bb6f278cfe4098/roles/4045191c71164a3e9a009f23ab3b1406` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:14:59.619294 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43aeaacc-9153-4a60-9011-d6f56e0deb54 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=140d8b9d73914a76b5bb6f278cfe4098, role_id=4045191c71164a3e9a009f23ab3b1406)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:14:59.620696 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43aeaacc-9153-4a60-9011-d6f56e0deb54 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:14:59.721280 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 162/334] 10.222.0.66 () {58 vars in 1334 bytes} [Mon Jul 18 02:14:59 2022] PUT /identity/v3/system/users/140d8b9d73914a76b5bb6f278cfe4098/roles/4045191c71164a3e9a009f23ab3b1406 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:15:01.122821 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d8e0a2-148b-439b-a375-599e3cc6d0d7 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:01.123789 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d8e0a2-148b-439b-a375-599e3cc6d0d7 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:01.124296 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d8e0a2-148b-439b-a375-599e3cc6d0d7 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:01.126074 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 173/335] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:15:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:15:01.144575 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1d58c4-2689-44d7-bda6-d6908c7f3257 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:01.145491 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1d58c4-2689-44d7-bda6-d6908c7f3257 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:01.145964 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1d58c4-2689-44d7-bda6-d6908c7f3257 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:01.151645 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:01.166879 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-0a1d58c4-2689-44d7-bda6-d6908c7f3257 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:15:01.174581 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-0a1d58c4-2689-44d7-bda6-d6908c7f3257 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:15:01.225444 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d78598-3bb0-48eb-ba15-553257148861 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:01.225780 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d78598-3bb0-48eb-ba15-553257148861 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:01.226007 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d78598-3bb0-48eb-ba15-553257148861 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:01.227673 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 174/336] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:15:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 02:15:01.238929 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9b99ea-3c03-4608-8329-6d77be3eb3be None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:01.239216 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9b99ea-3c03-4608-8329-6d77be3eb3be None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:01.239502 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9b99ea-3c03-4608-8329-6d77be3eb3be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:01.256289 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 163/337] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:15:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 02:15:01.268222 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-fc9b99ea-3c03-4608-8329-6d77be3eb3be None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:15:01.269430 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-fc9b99ea-3c03-4608-8329-6d77be3eb3be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:15:01.315669 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1d58c4-2689-44d7-bda6-d6908c7f3257 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:01.383029 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 175/338] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:15:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 02:15:01.455275 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9b99ea-3c03-4608-8329-6d77be3eb3be None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:01.474476 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:01.480133 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1d58c4-2689-44d7-bda6-d6908c7f3257 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:01.530515 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:01.533989 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8f21c3-4b93-4ef0-af98-6140dda54da4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:01.534837 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8f21c3-4b93-4ef0-af98-6140dda54da4 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:01.535121 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8f21c3-4b93-4ef0-af98-6140dda54da4 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:01.535396 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8f21c3-4b93-4ef0-af98-6140dda54da4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:01.552212 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8f21c3-4b93-4ef0-af98-6140dda54da4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 02:15:01.568316 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-0b8f21c3-4b93-4ef0-af98-6140dda54da4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 02:15:01.568316 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:01.568316 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:01.568316 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:01.568316 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:01.568316 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:01.568316 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:01.568316 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:01.568316 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:01.568316 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:01.568316 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:01.568316 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:01.568316 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 02:15:01.568316 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 02:15:01.568316 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 02:15:01.568316 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 02:15:01.569296 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 02:15:01.569296 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 02:15:01.569296 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 02:15:01.569296 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 02:15:01.569296 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:01.569296 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:01.569296 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:01.569296 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:01.569296 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:01.569296 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:01.569296 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:01.569296 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:01.569296 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:01.569296 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:01.569296 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:01.569296 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:01.570567 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:01.570567 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:01.570567 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:01.570567 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:01.570567 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 02:15:01.570567 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 02:15:01.570567 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 02:15:01.570567 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 02:15:01.570567 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 02:15:01.570567 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 02:15:01.570567 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 02:15:01.570567 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:15:01.570567 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 164/339] 10.222.0.66 () {56 vars in 1099 bytes} [Mon Jul 18 02:15:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 02:15:01.582372 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8f21c3-4b93-4ef0-af98-6140dda54da4 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:01.705389 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9b99ea-3c03-4608-8329-6d77be3eb3be None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:01.729029 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8f21c3-4b93-4ef0-af98-6140dda54da4 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:01.774819 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430c5f41-232a-4831-b731-c3af7b3171c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:01.775652 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430c5f41-232a-4831-b731-c3af7b3171c2 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:01.775941 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430c5f41-232a-4831-b731-c3af7b3171c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:01.776223 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430c5f41-232a-4831-b731-c3af7b3171c2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:01.792261 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430c5f41-232a-4831-b731-c3af7b3171c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 02:15:01.794789 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f39074-ffdc-4484-86c1-282f07662f55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:01.795480 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f39074-ffdc-4484-86c1-282f07662f55 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:01.795765 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f39074-ffdc-4484-86c1-282f07662f55 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:01.796044 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f39074-ffdc-4484-86c1-282f07662f55 None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:01.796807 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f39074-ffdc-4484-86c1-282f07662f55 None admin] RBAC: Adding query filter params (name=member) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:01.797224 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f39074-ffdc-4484-86c1-282f07662f55 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:01.798829 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f39074-ffdc-4484-86c1-282f07662f55 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:01.800936 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-430c5f41-232a-4831-b731-c3af7b3171c2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 02:15:01.800936 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:01.800936 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:01.800936 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:01.800936 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:01.800936 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:01.800936 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:01.800936 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:01.800936 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:01.800936 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:01.800936 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:01.800936 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:01.800936 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 02:15:01.800936 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 02:15:01.800936 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 02:15:01.800936 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 02:15:01.802613 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 02:15:01.802613 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 02:15:01.802613 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 02:15:01.802613 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 02:15:01.802613 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:01.802613 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:01.802613 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:01.802613 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:01.802613 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:01.802613 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:01.802613 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:01.802613 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:01.802613 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:01.802613 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:01.802613 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:01.802613 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:01.803793 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:01.803793 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:01.803793 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:01.803793 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:01.803793 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 02:15:01.803793 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 02:15:01.803793 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 02:15:01.803793 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 02:15:01.803793 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 02:15:01.803793 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 02:15:01.803793 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 02:15:01.803793 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:15:01.803793 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 176/340] 10.222.0.66 () {56 vars in 1114 bytes} [Mon Jul 18 02:15:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 348 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 02:15:01.808838 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430c5f41-232a-4831-b731-c3af7b3171c2 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:01.812857 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 165/341] 10.222.0.66 () {56 vars in 1101 bytes} [Mon Jul 18 02:15:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 02:15:01.819210 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f39074-ffdc-4484-86c1-282f07662f55 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:01.905911 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:01.946521 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:01.953189 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430c5f41-232a-4831-b731-c3af7b3171c2 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:02.011498 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e91ef37-a9af-43a1-84e4-b60eedc2eabd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:02.012106 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e91ef37-a9af-43a1-84e4-b60eedc2eabd None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:02.012394 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e91ef37-a9af-43a1-84e4-b60eedc2eabd None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:02.012713 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e91ef37-a9af-43a1-84e4-b60eedc2eabd None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:02.013433 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e91ef37-a9af-43a1-84e4-b60eedc2eabd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:02.013705 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f39074-ffdc-4484-86c1-282f07662f55 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:02.013885 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e91ef37-a9af-43a1-84e4-b60eedc2eabd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:02.015145 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e91ef37-a9af-43a1-84e4-b60eedc2eabd None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:02.028696 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 177/342] 10.222.0.66 () {56 vars in 1111 bytes} [Mon Jul 18 02:15:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:02.034699 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e91ef37-a9af-43a1-84e4-b60eedc2eabd None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:02.080280 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008f449f-04b7-489b-8062-51d20d7db5af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:02.080952 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008f449f-04b7-489b-8062-51d20d7db5af None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:02.081236 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008f449f-04b7-489b-8062-51d20d7db5af None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:02.081517 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008f449f-04b7-489b-8062-51d20d7db5af None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:02.150168 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e91ef37-a9af-43a1-84e4-b60eedc2eabd None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:02.189177 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899bbced-c4a6-44c2-9d76-a53958cb0b0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:02.189837 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899bbced-c4a6-44c2-9d76-a53958cb0b0e None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:02.190161 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899bbced-c4a6-44c2-9d76-a53958cb0b0e None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:02.190450 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899bbced-c4a6-44c2-9d76-a53958cb0b0e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:02.194423 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:02.199700 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008f449f-04b7-489b-8062-51d20d7db5af None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:02.201209 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008f449f-04b7-489b-8062-51d20d7db5af None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:02.266859 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899bbced-c4a6-44c2-9d76-a53958cb0b0e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:02.268352 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899bbced-c4a6-44c2-9d76-a53958cb0b0e None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:02.274500 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:02.339582 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 166/343] 10.222.0.66 () {58 vars in 1282 bytes} [Mon Jul 18 02:15:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 521 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 02:15:02.345991 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008f449f-04b7-489b-8062-51d20d7db5af None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:02.376649 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 178/344] 10.222.0.66 () {58 vars in 1282 bytes} [Mon Jul 18 02:15:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 18 02:15:02.382997 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899bbced-c4a6-44c2-9d76-a53958cb0b0e None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:02.434465 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:02.497994 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899bbced-c4a6-44c2-9d76-a53958cb0b0e None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:02.546536 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008f449f-04b7-489b-8062-51d20d7db5af None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:02.550820 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1beeec92-e101-4292-8a92-199c99ec9f35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:02.551701 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beeec92-e101-4292-8a92-199c99ec9f35 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:02.551983 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beeec92-e101-4292-8a92-199c99ec9f35 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:02.552257 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beeec92-e101-4292-8a92-199c99ec9f35 None admin] PATH_INFO: `/v3/projects/a5370165a2354ba6aec1bb43138f4838` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:02.553461 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1beeec92-e101-4292-8a92-199c99ec9f35 None admin] RBAC: Authorizing `identity:get_project(project_id=a5370165a2354ba6aec1bb43138f4838)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:02.554949 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1beeec92-e101-4292-8a92-199c99ec9f35 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:02.556988 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 179/345] 10.222.0.66 () {56 vars in 1186 bytes} [Mon Jul 18 02:15:02 2022] GET /identity/v3/projects/a5370165a2354ba6aec1bb43138f4838 => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:02.562235 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1beeec92-e101-4292-8a92-199c99ec9f35 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:02.622813 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c0d26a-629b-4d55-9278-e55ddafe6d5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:02.623702 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c0d26a-629b-4d55-9278-e55ddafe6d5b None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:02.623988 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c0d26a-629b-4d55-9278-e55ddafe6d5b None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:02.624265 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c0d26a-629b-4d55-9278-e55ddafe6d5b None admin] PATH_INFO: `/v3/projects/a5370165a2354ba6aec1bb43138f4838` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:02.630836 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c0d26a-629b-4d55-9278-e55ddafe6d5b None admin] RBAC: Authorizing `identity:get_project(project_id=a5370165a2354ba6aec1bb43138f4838)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:02.632307 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c0d26a-629b-4d55-9278-e55ddafe6d5b None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:02.634411 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 167/346] 10.222.0.66 () {56 vars in 1186 bytes} [Mon Jul 18 02:15:02 2022] GET /identity/v3/projects/a5370165a2354ba6aec1bb43138f4838 => generated 298 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 02:15:02.635636 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba3cb0b-978d-4cc6-9615-9bc329b5b62e None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:02.635928 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba3cb0b-978d-4cc6-9615-9bc329b5b62e None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:02.636204 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba3cb0b-978d-4cc6-9615-9bc329b5b62e None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:02.637689 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 168/347] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:15:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:15:02.643045 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba3cb0b-978d-4cc6-9615-9bc329b5b62e None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:02.680822 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:02.688900 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:02.694542 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1beeec92-e101-4292-8a92-199c99ec9f35 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:02.735360 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc64e8b-e478-409b-b7c7-d8a67e2bf965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:02.736229 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc64e8b-e478-409b-b7c7-d8a67e2bf965 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:02.736508 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc64e8b-e478-409b-b7c7-d8a67e2bf965 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:02.736779 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc64e8b-e478-409b-b7c7-d8a67e2bf965 None admin] PATH_INFO: `/v3/groups/02deab00a40e4493bb6ddac19fc25741` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:02.747471 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc64e8b-e478-409b-b7c7-d8a67e2bf965 None admin] RBAC: Authorizing `identity:get_group(group_id=02deab00a40e4493bb6ddac19fc25741)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:02.748933 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc64e8b-e478-409b-b7c7-d8a67e2bf965 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:02.750952 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 180/348] 10.222.0.66 () {56 vars in 1180 bytes} [Mon Jul 18 02:15:02 2022] GET /identity/v3/groups/02deab00a40e4493bb6ddac19fc25741 => generated 226 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:02.751973 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997a4948-b167-4810-bf59-fcba553e728c None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:02.752258 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997a4948-b167-4810-bf59-fcba553e728c None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:02.752529 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997a4948-b167-4810-bf59-fcba553e728c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:02.789280 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-997a4948-b167-4810-bf59-fcba553e728c None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:15:02.790496 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-997a4948-b167-4810-bf59-fcba553e728c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:15:02.839336 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba3cb0b-978d-4cc6-9615-9bc329b5b62e None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:02.895800 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6140d06c-5179-4a34-ab44-54c9b9a0c8c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:02.897157 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6140d06c-5179-4a34-ab44-54c9b9a0c8c3 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:02.897437 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6140d06c-5179-4a34-ab44-54c9b9a0c8c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:02.897710 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6140d06c-5179-4a34-ab44-54c9b9a0c8c3 None admin] PATH_INFO: `/v3/groups/02deab00a40e4493bb6ddac19fc25741` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:02.899746 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6140d06c-5179-4a34-ab44-54c9b9a0c8c3 None admin] RBAC: Authorizing `identity:get_group(group_id=02deab00a40e4493bb6ddac19fc25741)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:02.901201 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6140d06c-5179-4a34-ab44-54c9b9a0c8c3 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:02.903183 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 169/349] 10.222.0.66 () {56 vars in 1180 bytes} [Mon Jul 18 02:15:02 2022] GET /identity/v3/groups/02deab00a40e4493bb6ddac19fc25741 => generated 226 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:02.904556 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6140d06c-5179-4a34-ab44-54c9b9a0c8c3 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:02.945648 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 181/350] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:15:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 02:15:02.946534 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997a4948-b167-4810-bf59-fcba553e728c None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:03.029654 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:03.044888 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6140d06c-5179-4a34-ab44-54c9b9a0c8c3 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:03.085869 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:03.092997 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d1321a-dbc3-4cc5-a452-6498134eeaaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:03.093731 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d1321a-dbc3-4cc5-a452-6498134eeaaf None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:03.094476 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d1321a-dbc3-4cc5-a452-6498134eeaaf None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:03.094760 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d1321a-dbc3-4cc5-a452-6498134eeaaf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:03.101015 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d1321a-dbc3-4cc5-a452-6498134eeaaf None admin] RBAC: Adding query filter params (group.id=02deab00a40e4493bb6ddac19fc25741, role.id=a5b895323e7b4f0db2491abe9ad57ee9, scope.project.id=a5370165a2354ba6aec1bb43138f4838) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:03.101015 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d1321a-dbc3-4cc5-a452-6498134eeaaf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:03.101015 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d1321a-dbc3-4cc5-a452-6498134eeaaf None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:03.104234 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997a4948-b167-4810-bf59-fcba553e728c None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:03.104656 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 170/351] 10.222.0.66 () {56 vars in 1376 bytes} [Mon Jul 18 02:15:02 2022] GET /identity/v3/role_assignments?group.id=02deab00a40e4493bb6ddac19fc25741&scope.project.id=a5370165a2354ba6aec1bb43138f4838&role.id=a5b895323e7b4f0db2491abe9ad57ee9 => generated 260 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:03.109096 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d1321a-dbc3-4cc5-a452-6498134eeaaf None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:03.231787 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40a549c-df0a-4169-9e29-712e5e563054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:03.232521 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40a549c-df0a-4169-9e29-712e5e563054 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:03.232756 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40a549c-df0a-4169-9e29-712e5e563054 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:03.233036 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40a549c-df0a-4169-9e29-712e5e563054 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:03.233887 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40a549c-df0a-4169-9e29-712e5e563054 None admin] RBAC: Adding query filter params (group.id=02deab00a40e4493bb6ddac19fc25741, role.id=65feabf3b27944a8b01a86c8ac608457, scope.project.id=a5370165a2354ba6aec1bb43138f4838) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:03.234230 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40a549c-df0a-4169-9e29-712e5e563054 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:03.236846 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40a549c-df0a-4169-9e29-712e5e563054 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:03.249395 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 182/352] 10.222.0.66 () {56 vars in 1376 bytes} [Mon Jul 18 02:15:02 2022] GET /identity/v3/role_assignments?group.id=02deab00a40e4493bb6ddac19fc25741&scope.project.id=a5370165a2354ba6aec1bb43138f4838&role.id=65feabf3b27944a8b01a86c8ac608457 => generated 260 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:03.251247 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83799f3-97f0-4230-8091-3675c8bc8622 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:03.251795 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83799f3-97f0-4230-8091-3675c8bc8622 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:03.252323 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83799f3-97f0-4230-8091-3675c8bc8622 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:03.254273 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 183/353] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:15:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:15:03.255802 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495566ad-208f-4893-9c36-0ea5805cfa19 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:03.256313 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495566ad-208f-4893-9c36-0ea5805cfa19 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:03.256784 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495566ad-208f-4893-9c36-0ea5805cfa19 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:03.262563 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 184/354] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:15:03 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:15:03.263036 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d1321a-dbc3-4cc5-a452-6498134eeaaf None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:03.264915 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047ee8a9-9768-4506-a552-ff0362f33f9e None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:03.265424 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047ee8a9-9768-4506-a552-ff0362f33f9e None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:03.266552 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047ee8a9-9768-4506-a552-ff0362f33f9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:03.297648 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-047ee8a9-9768-4506-a552-ff0362f33f9e None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:15:03.300619 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-047ee8a9-9768-4506-a552-ff0362f33f9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:15:03.317491 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdda585-9c48-4d0a-80ad-e3bb2471bc6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:03.318351 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdda585-9c48-4d0a-80ad-e3bb2471bc6d None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:03.318639 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdda585-9c48-4d0a-80ad-e3bb2471bc6d None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:03.318927 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdda585-9c48-4d0a-80ad-e3bb2471bc6d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:03.344476 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdda585-9c48-4d0a-80ad-e3bb2471bc6d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 02:15:03.353656 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-2cdda585-9c48-4d0a-80ad-e3bb2471bc6d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 02:15:03.353656 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:03.353656 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:03.353656 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:03.353656 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:03.353656 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:03.353656 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:03.353656 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:03.353656 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:03.353656 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:03.353656 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:03.353656 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:03.353656 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 02:15:03.353656 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 02:15:03.353656 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 02:15:03.353656 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 02:15:03.354699 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 02:15:03.354699 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 02:15:03.354699 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 02:15:03.354699 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 02:15:03.354699 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:03.354699 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:03.354699 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:03.354699 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:03.354699 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:03.354699 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:03.354699 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:03.354699 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:03.354699 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:03.354699 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:03.354699 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:03.354699 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:03.355544 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:03.355544 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:03.355544 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:03.355544 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:03.355544 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 02:15:03.355544 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 02:15:03.355544 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 02:15:03.355544 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 02:15:03.355544 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 02:15:03.355544 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 02:15:03.355544 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 02:15:03.355544 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:15:03.355544 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 171/355] 10.222.0.66 () {56 vars in 1099 bytes} [Mon Jul 18 02:15:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:03.356235 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd189820-4db2-4b4c-a3c5-76558c31f9ca None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:03.356845 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd189820-4db2-4b4c-a3c5-76558c31f9ca None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:03.356845 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd189820-4db2-4b4c-a3c5-76558c31f9ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:03.381919 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-bd189820-4db2-4b4c-a3c5-76558c31f9ca None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:15:03.383213 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-bd189820-4db2-4b4c-a3c5-76558c31f9ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:15:03.397161 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:03.400281 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 185/356] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:15:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 02:15:03.401716 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047ee8a9-9768-4506-a552-ff0362f33f9e None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:03.476384 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 172/357] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:15:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 02:15:03.478624 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383ee636-8661-4778-a3e0-2e948fb35aab None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:03.479218 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383ee636-8661-4778-a3e0-2e948fb35aab None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:03.479696 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383ee636-8661-4778-a3e0-2e948fb35aab None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:03.481355 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 173/358] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:15:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:15:03.482374 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383ee636-8661-4778-a3e0-2e948fb35aab None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:03.542179 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:03.566409 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047ee8a9-9768-4506-a552-ff0362f33f9e None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:03.617787 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383ee636-8661-4778-a3e0-2e948fb35aab None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:03.623592 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71f7f46-e0e7-4db8-bfb7-461832a978b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:03.624270 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71f7f46-e0e7-4db8-bfb7-461832a978b9 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:03.624553 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71f7f46-e0e7-4db8-bfb7-461832a978b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:03.624825 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71f7f46-e0e7-4db8-bfb7-461832a978b9 None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:03.627200 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71f7f46-e0e7-4db8-bfb7-461832a978b9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:03.627549 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71f7f46-e0e7-4db8-bfb7-461832a978b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:03.629448 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71f7f46-e0e7-4db8-bfb7-461832a978b9 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:03.641474 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 186/359] 10.222.0.66 () {56 vars in 1101 bytes} [Mon Jul 18 02:15:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:03.642896 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfe7afc-db1e-4b98-ab41-0bd90bba8e6b None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:03.643189 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfe7afc-db1e-4b98-ab41-0bd90bba8e6b None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:03.643471 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfe7afc-db1e-4b98-ab41-0bd90bba8e6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:03.661544 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-1bfe7afc-db1e-4b98-ab41-0bd90bba8e6b None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:15:03.662885 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-1bfe7afc-db1e-4b98-ab41-0bd90bba8e6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:15:03.672801 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e69831-03af-4b4d-b99f-30d73e4f0b5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:03.674267 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e69831-03af-4b4d-b99f-30d73e4f0b5d None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:03.674554 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e69831-03af-4b4d-b99f-30d73e4f0b5d None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:03.674832 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e69831-03af-4b4d-b99f-30d73e4f0b5d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:03.689317 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e69831-03af-4b4d-b99f-30d73e4f0b5d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 02:15:03.699643 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-e3e69831-03af-4b4d-b99f-30d73e4f0b5d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 02:15:03.699643 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:03.699643 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:03.699643 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:03.699643 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:03.699643 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:03.699643 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:03.699643 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:03.699643 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:03.699643 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:03.699643 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:03.699643 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:03.699643 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 02:15:03.699643 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 02:15:03.699643 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 02:15:03.699643 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 02:15:03.700596 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 02:15:03.700596 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 02:15:03.700596 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 02:15:03.700596 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 02:15:03.700596 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:03.700596 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:03.700596 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:03.700596 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:03.700596 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:03.700596 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:03.700596 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:03.700596 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:03.700596 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:03.700596 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:03.700596 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:03.700596 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:03.701902 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:03.701902 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:03.701902 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:03.701902 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:03.701902 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 02:15:03.701902 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 02:15:03.701902 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 02:15:03.701902 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 02:15:03.701902 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 02:15:03.701902 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 02:15:03.701902 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 02:15:03.701902 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:15:03.701902 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 174/360] 10.222.0.66 () {56 vars in 1099 bytes} [Mon Jul 18 02:15:03 2022] GET /identity/v3/roles/reader => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:03.702863 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e69831-03af-4b4d-b99f-30d73e4f0b5d None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:03.707140 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:03.742262 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 187/361] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:15:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 02:15:03.743414 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfe7afc-db1e-4b98-ab41-0bd90bba8e6b None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:03.835175 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e69831-03af-4b4d-b99f-30d73e4f0b5d None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:03.841309 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:03.870841 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfe7afc-db1e-4b98-ab41-0bd90bba8e6b None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:03.883262 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f71172-ecbb-4905-8f74-d5b8d13d5012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:03.884116 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f71172-ecbb-4905-8f74-d5b8d13d5012 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:03.884415 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f71172-ecbb-4905-8f74-d5b8d13d5012 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:03.885053 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f71172-ecbb-4905-8f74-d5b8d13d5012 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:03.907142 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f71172-ecbb-4905-8f74-d5b8d13d5012 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 02:15:03.917795 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-a9f71172-ecbb-4905-8f74-d5b8d13d5012 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 02:15:03.917795 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:03.917795 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:03.917795 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:03.917795 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:03.917795 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:03.917795 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:03.917795 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:03.917795 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:03.917795 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:03.917795 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:03.917795 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:03.917795 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 02:15:03.917795 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 02:15:03.917795 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 02:15:03.917795 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 02:15:03.918795 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 02:15:03.918795 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 02:15:03.918795 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 02:15:03.918795 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 02:15:03.918795 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:03.918795 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:03.918795 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:03.918795 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:03.918795 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:03.918795 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:03.918795 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:03.918795 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:03.918795 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:03.918795 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:03.918795 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:03.918795 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:03.919616 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:03.919616 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:03.919616 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:03.919616 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:03.919616 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 02:15:03.919616 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 02:15:03.919616 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 02:15:03.919616 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 02:15:03.919616 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 02:15:03.919616 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 02:15:03.919616 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 02:15:03.919616 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:15:03.919616 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 175/362] 10.222.0.66 () {56 vars in 1096 bytes} [Mon Jul 18 02:15:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:03.920804 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f71172-ecbb-4905-8f74-d5b8d13d5012 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:03.930896 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f856d17-30d5-42ff-805d-8b4adbd144cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:03.931871 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f856d17-30d5-42ff-805d-8b4adbd144cf None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:03.932374 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f856d17-30d5-42ff-805d-8b4adbd144cf None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:03.932841 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f856d17-30d5-42ff-805d-8b4adbd144cf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:04.011661 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:04.033776 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f856d17-30d5-42ff-805d-8b4adbd144cf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:04.036059 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f856d17-30d5-42ff-805d-8b4adbd144cf None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:04.059485 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f71172-ecbb-4905-8f74-d5b8d13d5012 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:04.098375 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc34064-a7d2-49b0-b757-c8f8657b9747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:04.098809 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc34064-a7d2-49b0-b757-c8f8657b9747 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:04.099097 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc34064-a7d2-49b0-b757-c8f8657b9747 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:04.099378 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc34064-a7d2-49b0-b757-c8f8657b9747 None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:04.100134 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc34064-a7d2-49b0-b757-c8f8657b9747 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:04.100444 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc34064-a7d2-49b0-b757-c8f8657b9747 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:04.101847 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc34064-a7d2-49b0-b757-c8f8657b9747 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:04.112514 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 176/363] 10.222.0.66 () {56 vars in 1101 bytes} [Mon Jul 18 02:15:03 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:04.112865 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc34064-a7d2-49b0-b757-c8f8657b9747 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:04.128349 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:04.152500 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 188/364] 10.222.0.66 () {58 vars in 1282 bytes} [Mon Jul 18 02:15:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 02:15:04.153663 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f856d17-30d5-42ff-805d-8b4adbd144cf None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:04.230100 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc34064-a7d2-49b0-b757-c8f8657b9747 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:04.260530 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f856d17-30d5-42ff-805d-8b4adbd144cf None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:04.261046 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:04.272372 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797f5570-9f15-4979-a834-148e68b5bb9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:04.272988 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797f5570-9f15-4979-a834-148e68b5bb9a None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:04.274154 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797f5570-9f15-4979-a834-148e68b5bb9a None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:04.274154 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797f5570-9f15-4979-a834-148e68b5bb9a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:04.284915 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797f5570-9f15-4979-a834-148e68b5bb9a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 02:15:04.294354 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-797f5570-9f15-4979-a834-148e68b5bb9a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 02:15:04.294354 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:04.294354 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:04.294354 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:04.294354 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:04.294354 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:04.294354 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:04.294354 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:04.294354 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:04.294354 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:04.294354 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:04.294354 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:04.294354 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 02:15:04.294354 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 02:15:04.294354 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 02:15:04.294354 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 02:15:04.295319 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 02:15:04.295319 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 02:15:04.295319 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 02:15:04.295319 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 02:15:04.295319 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:04.295319 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:04.295319 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:04.295319 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:04.295319 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:04.295319 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:04.295319 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:04.295319 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:04.295319 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:04.295319 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:04.295319 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:04.295319 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:04.296127 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:04.296127 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:04.296127 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:04.296127 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:04.296127 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 02:15:04.296127 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 02:15:04.296127 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 02:15:04.296127 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 02:15:04.296127 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 02:15:04.296127 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 02:15:04.296127 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 02:15:04.296127 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:15:04.296127 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 177/365] 10.222.0.66 () {56 vars in 1114 bytes} [Mon Jul 18 02:15:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:04.297312 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797f5570-9f15-4979-a834-148e68b5bb9a None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:04.303994 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31fbf85-59a3-46df-92ac-db3b4aead03f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:04.305004 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31fbf85-59a3-46df-92ac-db3b4aead03f None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:04.305480 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31fbf85-59a3-46df-92ac-db3b4aead03f None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:04.305930 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31fbf85-59a3-46df-92ac-db3b4aead03f None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:04.306909 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31fbf85-59a3-46df-92ac-db3b4aead03f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:04.307984 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31fbf85-59a3-46df-92ac-db3b4aead03f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:04.309547 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31fbf85-59a3-46df-92ac-db3b4aead03f None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:04.319344 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 189/366] 10.222.0.66 () {56 vars in 1099 bytes} [Mon Jul 18 02:15:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:04.320682 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31fbf85-59a3-46df-92ac-db3b4aead03f None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:04.372566 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:04.417964 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797f5570-9f15-4979-a834-148e68b5bb9a None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:04.432411 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31fbf85-59a3-46df-92ac-db3b4aead03f None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:04.457614 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba352add-8439-4336-97fa-a72bf4b67e8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:04.458558 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba352add-8439-4336-97fa-a72bf4b67e8e None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:04.458845 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba352add-8439-4336-97fa-a72bf4b67e8e None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:04.459441 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba352add-8439-4336-97fa-a72bf4b67e8e None admin] PATH_INFO: `/v3/users/140d8b9d73914a76b5bb6f278cfe4098` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:04.460534 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba352add-8439-4336-97fa-a72bf4b67e8e None admin] RBAC: Authorizing `identity:get_user(user_id=140d8b9d73914a76b5bb6f278cfe4098)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:04.461953 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba352add-8439-4336-97fa-a72bf4b67e8e None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:04.464057 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 178/367] 10.222.0.66 () {56 vars in 1177 bytes} [Mon Jul 18 02:15:04 2022] GET /identity/v3/users/140d8b9d73914a76b5bb6f278cfe4098 => generated 293 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:04.465104 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba352add-8439-4336-97fa-a72bf4b67e8e None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:04.475938 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b41148f-65f7-4a05-8244-d4901aef8ed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:04.477168 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b41148f-65f7-4a05-8244-d4901aef8ed7 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:04.477715 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b41148f-65f7-4a05-8244-d4901aef8ed7 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:04.478222 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b41148f-65f7-4a05-8244-d4901aef8ed7 None admin] PATH_INFO: `/v3/projects/9748772803a44ccea6827637294a3a97` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:04.479675 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b41148f-65f7-4a05-8244-d4901aef8ed7 None admin] RBAC: Authorizing `identity:get_project(project_id=9748772803a44ccea6827637294a3a97)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:04.481345 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b41148f-65f7-4a05-8244-d4901aef8ed7 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:04.483345 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 190/368] 10.222.0.66 () {56 vars in 1186 bytes} [Mon Jul 18 02:15:04 2022] GET /identity/v3/projects/9748772803a44ccea6827637294a3a97 => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:04.485042 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b41148f-65f7-4a05-8244-d4901aef8ed7 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:04.533859 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:04.586096 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba352add-8439-4336-97fa-a72bf4b67e8e None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:04.612148 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b41148f-65f7-4a05-8244-d4901aef8ed7 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:04.630405 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:04.646983 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c434e758-3f75-4756-a242-2ed2150914bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:04.649411 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c434e758-3f75-4756-a242-2ed2150914bc None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:04.649907 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c434e758-3f75-4756-a242-2ed2150914bc None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:04.650392 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c434e758-3f75-4756-a242-2ed2150914bc None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:04.651408 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c434e758-3f75-4756-a242-2ed2150914bc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:04.651924 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c434e758-3f75-4756-a242-2ed2150914bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:04.653503 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c434e758-3f75-4756-a242-2ed2150914bc None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:04.657768 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4898d3d-4991-4f4b-8469-86ef66bc6988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:04.658417 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4898d3d-4991-4f4b-8469-86ef66bc6988 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:04.658705 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4898d3d-4991-4f4b-8469-86ef66bc6988 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:04.659277 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4898d3d-4991-4f4b-8469-86ef66bc6988 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:04.666274 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 179/369] 10.222.0.66 () {56 vars in 1111 bytes} [Mon Jul 18 02:15:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:04.667465 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c434e758-3f75-4756-a242-2ed2150914bc None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:04.736048 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4898d3d-4991-4f4b-8469-86ef66bc6988 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:04.737472 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4898d3d-4991-4f4b-8469-86ef66bc6988 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:04.789800 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c434e758-3f75-4756-a242-2ed2150914bc None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:04.815899 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:04.833674 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2494019-3600-4c18-b1c0-9e521d67cfad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:04.834687 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2494019-3600-4c18-b1c0-9e521d67cfad None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:04.835336 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2494019-3600-4c18-b1c0-9e521d67cfad None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:04.835943 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2494019-3600-4c18-b1c0-9e521d67cfad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:04.838149 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2494019-3600-4c18-b1c0-9e521d67cfad None admin] RBAC: Adding query filter params (role.id=4045191c71164a3e9a009f23ab3b1406, user.id=140d8b9d73914a76b5bb6f278cfe4098, scope.system=all) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:04.838799 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2494019-3600-4c18-b1c0-9e521d67cfad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:04.840460 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2494019-3600-4c18-b1c0-9e521d67cfad None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:04.842223 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 191/370] 10.222.0.66 () {58 vars in 1282 bytes} [Mon Jul 18 02:15:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 02:15:04.843055 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4898d3d-4991-4f4b-8469-86ef66bc6988 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:04.853287 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:04.855969 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 180/371] 10.222.0.66 () {56 vars in 1308 bytes} [Mon Jul 18 02:15:04 2022] GET /identity/v3/role_assignments?user.id=140d8b9d73914a76b5bb6f278cfe4098&scope.system=all&role.id=4045191c71164a3e9a009f23ab3b1406 => generated 510 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:04.857010 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2494019-3600-4c18-b1c0-9e521d67cfad None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:04.954570 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4898d3d-4991-4f4b-8469-86ef66bc6988 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:04.981168 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2494019-3600-4c18-b1c0-9e521d67cfad None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:04.991330 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a9088c-0f77-4c7a-bca5-ac4d1b1f3b7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:04.992132 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a9088c-0f77-4c7a-bca5-ac4d1b1f3b7e None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:04.992419 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a9088c-0f77-4c7a-bca5-ac4d1b1f3b7e None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:04.992697 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a9088c-0f77-4c7a-bca5-ac4d1b1f3b7e None admin] PATH_INFO: `/v3/groups/02deab00a40e4493bb6ddac19fc25741` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:04.993978 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a9088c-0f77-4c7a-bca5-ac4d1b1f3b7e None admin] RBAC: Authorizing `identity:get_group(group_id=02deab00a40e4493bb6ddac19fc25741)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:04.995477 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a9088c-0f77-4c7a-bca5-ac4d1b1f3b7e None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:04.997665 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 192/372] 10.222.0.66 () {56 vars in 1180 bytes} [Mon Jul 18 02:15:04 2022] GET /identity/v3/groups/02deab00a40e4493bb6ddac19fc25741 => generated 226 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:04.997912 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a9088c-0f77-4c7a-bca5-ac4d1b1f3b7e None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:05.041715 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd521a93-7d4a-4d60-9223-ea195a3a4ed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:05.042412 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd521a93-7d4a-4d60-9223-ea195a3a4ed0 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:05.042696 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd521a93-7d4a-4d60-9223-ea195a3a4ed0 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:05.042969 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd521a93-7d4a-4d60-9223-ea195a3a4ed0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:05.100419 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:05.105821 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:05.120394 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a9088c-0f77-4c7a-bca5-ac4d1b1f3b7e None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:05.155716 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd521a93-7d4a-4d60-9223-ea195a3a4ed0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:05.158233 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2258184-83d4-45d4-a57d-23ea29c26d25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:05.159696 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd521a93-7d4a-4d60-9223-ea195a3a4ed0 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:05.159808 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2258184-83d4-45d4-a57d-23ea29c26d25 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:05.159808 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2258184-83d4-45d4-a57d-23ea29c26d25 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:05.159808 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2258184-83d4-45d4-a57d-23ea29c26d25 None admin] PATH_INFO: `/v3/projects/a05e40a15656441b94f5763e4d68d305` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:05.160846 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2258184-83d4-45d4-a57d-23ea29c26d25 None admin] RBAC: Authorizing `identity:get_project(project_id=a05e40a15656441b94f5763e4d68d305)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:05.162328 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2258184-83d4-45d4-a57d-23ea29c26d25 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:05.165474 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 193/373] 10.222.0.66 () {56 vars in 1186 bytes} [Mon Jul 18 02:15:04 2022] GET /identity/v3/projects/a05e40a15656441b94f5763e4d68d305 => generated 344 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:05.166408 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2258184-83d4-45d4-a57d-23ea29c26d25 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:05.292496 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2258184-83d4-45d4-a57d-23ea29c26d25 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:05.303727 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 181/374] 10.222.0.66 () {58 vars in 1282 bytes} [Mon Jul 18 02:15:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 02:15:05.305394 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd521a93-7d4a-4d60-9223-ea195a3a4ed0 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:05.335101 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79241a0e-33db-489b-b0e0-b531e22a470e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:05.335759 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79241a0e-33db-489b-b0e0-b531e22a470e None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:05.336053 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79241a0e-33db-489b-b0e0-b531e22a470e None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:05.336777 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79241a0e-33db-489b-b0e0-b531e22a470e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:05.337898 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79241a0e-33db-489b-b0e0-b531e22a470e None admin] RBAC: Adding query filter params (group.id=02deab00a40e4493bb6ddac19fc25741, role.id=65feabf3b27944a8b01a86c8ac608457, scope.project.id=9748772803a44ccea6827637294a3a97) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:05.338263 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79241a0e-33db-489b-b0e0-b531e22a470e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:05.339912 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79241a0e-33db-489b-b0e0-b531e22a470e None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:05.352648 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 194/375] 10.222.0.66 () {56 vars in 1376 bytes} [Mon Jul 18 02:15:05 2022] GET /identity/v3/role_assignments?group.id=02deab00a40e4493bb6ddac19fc25741&scope.project.id=9748772803a44ccea6827637294a3a97&role.id=65feabf3b27944a8b01a86c8ac608457 => generated 260 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:05.352648 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79241a0e-33db-489b-b0e0-b531e22a470e None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:05.366662 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:05.436259 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:05.470517 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79241a0e-33db-489b-b0e0-b531e22a470e None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:05.504031 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd521a93-7d4a-4d60-9223-ea195a3a4ed0 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:05.513805 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5206a568-b628-4d08-8ac0-fd956f34946f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:05.514712 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5206a568-b628-4d08-8ac0-fd956f34946f None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:05.514996 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5206a568-b628-4d08-8ac0-fd956f34946f None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:05.515270 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5206a568-b628-4d08-8ac0-fd956f34946f None admin] PATH_INFO: `/v3/projects/9748772803a44ccea6827637294a3a97` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:05.516932 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5206a568-b628-4d08-8ac0-fd956f34946f None admin] RBAC: Authorizing `identity:get_project(project_id=9748772803a44ccea6827637294a3a97)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:05.518431 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5206a568-b628-4d08-8ac0-fd956f34946f None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:05.524630 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 195/376] 10.222.0.66 () {56 vars in 1186 bytes} [Mon Jul 18 02:15:05 2022] GET /identity/v3/projects/9748772803a44ccea6827637294a3a97 => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:05.526741 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5206a568-b628-4d08-8ac0-fd956f34946f None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:05.574734 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabbc0d7-403d-4405-aba7-4ee6ffee4c39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:05.575538 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabbc0d7-403d-4405-aba7-4ee6ffee4c39 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:05.575823 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabbc0d7-403d-4405-aba7-4ee6ffee4c39 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:05.576103 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabbc0d7-403d-4405-aba7-4ee6ffee4c39 None admin] PATH_INFO: `/v3/groups/e429b16d0a744db79625f143ae9e4696` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:05.592468 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabbc0d7-403d-4405-aba7-4ee6ffee4c39 None admin] RBAC: Authorizing `identity:get_group(group_id=e429b16d0a744db79625f143ae9e4696)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:05.594545 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabbc0d7-403d-4405-aba7-4ee6ffee4c39 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:05.606752 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 182/377] 10.222.0.66 () {56 vars in 1180 bytes} [Mon Jul 18 02:15:05 2022] GET /identity/v3/groups/e429b16d0a744db79625f143ae9e4696 => generated 229 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:05.624174 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabbc0d7-403d-4405-aba7-4ee6ffee4c39 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:05.692260 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:05.736935 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5206a568-b628-4d08-8ac0-fd956f34946f None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:05.751390 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabbc0d7-403d-4405-aba7-4ee6ffee4c39 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:05.780299 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1e8a53-4d10-4787-9231-2c6a68cc026a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:05.781732 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1e8a53-4d10-4787-9231-2c6a68cc026a None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:05.782008 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1e8a53-4d10-4787-9231-2c6a68cc026a None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:05.782306 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1e8a53-4d10-4787-9231-2c6a68cc026a None admin] PATH_INFO: `/v3/groups/02deab00a40e4493bb6ddac19fc25741` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:05.783490 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1e8a53-4d10-4787-9231-2c6a68cc026a None admin] RBAC: Authorizing `identity:get_group(group_id=02deab00a40e4493bb6ddac19fc25741)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:05.784939 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1e8a53-4d10-4787-9231-2c6a68cc026a None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:05.786855 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 196/378] 10.222.0.66 () {56 vars in 1180 bytes} [Mon Jul 18 02:15:05 2022] GET /identity/v3/groups/02deab00a40e4493bb6ddac19fc25741 => generated 226 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:05.790253 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6d43c2-8460-4768-8157-e636d152e4fe None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:05.790253 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6d43c2-8460-4768-8157-e636d152e4fe None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:05.790253 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6d43c2-8460-4768-8157-e636d152e4fe None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:05.790253 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 197/379] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:15:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:15:05.791001 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84f8c81-82a2-4d5e-88ce-386fc7ef57fb None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:05.791280 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84f8c81-82a2-4d5e-88ce-386fc7ef57fb None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:05.791551 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84f8c81-82a2-4d5e-88ce-386fc7ef57fb None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:05.792771 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 198/380] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:15:05 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:15:05.793460 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84f8c81-82a2-4d5e-88ce-386fc7ef57fb None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:05.800270 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5681eecf-2bd6-4ebf-80f9-3ac7ea29cbed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:05.801110 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5681eecf-2bd6-4ebf-80f9-3ac7ea29cbed None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:05.801404 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5681eecf-2bd6-4ebf-80f9-3ac7ea29cbed None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:05.801688 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5681eecf-2bd6-4ebf-80f9-3ac7ea29cbed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:05.802586 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5681eecf-2bd6-4ebf-80f9-3ac7ea29cbed None admin] RBAC: Adding query filter params (group.id=e429b16d0a744db79625f143ae9e4696, role.id=d4aa3324f0c84887b41d0dc4acf7f30e, scope.project.id=a05e40a15656441b94f5763e4d68d305) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:05.802906 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5681eecf-2bd6-4ebf-80f9-3ac7ea29cbed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:05.804367 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5681eecf-2bd6-4ebf-80f9-3ac7ea29cbed None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:05.810881 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 183/381] 10.222.0.66 () {56 vars in 1376 bytes} [Mon Jul 18 02:15:05 2022] GET /identity/v3/role_assignments?group.id=e429b16d0a744db79625f143ae9e4696&scope.project.id=a05e40a15656441b94f5763e4d68d305&role.id=d4aa3324f0c84887b41d0dc4acf7f30e => generated 260 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 02:15:05.812218 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0abc3a0-b87a-4d8a-a6b7-a98528887655 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:05.812550 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0abc3a0-b87a-4d8a-a6b7-a98528887655 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:05.812835 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0abc3a0-b87a-4d8a-a6b7-a98528887655 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:05.817930 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:05.833394 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-b0abc3a0-b87a-4d8a-a6b7-a98528887655 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:15:05.833394 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-b0abc3a0-b87a-4d8a-a6b7-a98528887655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:15:05.912941 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 184/382] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:15:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 02:15:05.914208 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae046e78-968a-4331-8cdc-740f10138098 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:05.914505 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae046e78-968a-4331-8cdc-740f10138098 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:05.914787 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae046e78-968a-4331-8cdc-740f10138098 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:05.940133 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84f8c81-82a2-4d5e-88ce-386fc7ef57fb None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:05.952070 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-ae046e78-968a-4331-8cdc-740f10138098 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:15:05.954523 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-ae046e78-968a-4331-8cdc-740f10138098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:15:05.991997 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78988438-107b-4561-bd29-24bfc9a088e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:05.992944 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78988438-107b-4561-bd29-24bfc9a088e0 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:05.993234 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78988438-107b-4561-bd29-24bfc9a088e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:05.993515 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78988438-107b-4561-bd29-24bfc9a088e0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:05.994908 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78988438-107b-4561-bd29-24bfc9a088e0 None admin] RBAC: Adding query filter params (group.id=02deab00a40e4493bb6ddac19fc25741, role.id=a5b895323e7b4f0db2491abe9ad57ee9, scope.project.id=9748772803a44ccea6827637294a3a97) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:05.995243 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78988438-107b-4561-bd29-24bfc9a088e0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:05.997034 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78988438-107b-4561-bd29-24bfc9a088e0 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:06.004270 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 199/383] 10.222.0.66 () {56 vars in 1376 bytes} [Mon Jul 18 02:15:05 2022] GET /identity/v3/role_assignments?group.id=02deab00a40e4493bb6ddac19fc25741&scope.project.id=9748772803a44ccea6827637294a3a97&role.id=a5b895323e7b4f0db2491abe9ad57ee9 => generated 260 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:06.005119 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78988438-107b-4561-bd29-24bfc9a088e0 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:06.034440 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:06.096059 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 185/384] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:15:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 02:15:06.125123 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae046e78-968a-4331-8cdc-740f10138098 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:06.206178 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:06.244274 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78988438-107b-4561-bd29-24bfc9a088e0 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:06.253484 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae046e78-968a-4331-8cdc-740f10138098 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:06.292844 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14fc1e6-4940-410c-b327-a5346cbab180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:06.294282 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14fc1e6-4940-410c-b327-a5346cbab180 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:06.294933 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14fc1e6-4940-410c-b327-a5346cbab180 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:06.295400 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14fc1e6-4940-410c-b327-a5346cbab180 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:06.307896 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14fc1e6-4940-410c-b327-a5346cbab180 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 02:15:06.318379 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-b14fc1e6-4940-410c-b327-a5346cbab180 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 02:15:06.318379 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:06.318379 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:06.318379 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:06.318379 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:06.318379 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:06.318379 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:06.318379 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:06.318379 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:06.318379 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:06.318379 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:06.318379 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:06.318379 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 02:15:06.318379 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 02:15:06.318379 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 02:15:06.318379 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 02:15:06.319319 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 02:15:06.319319 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 02:15:06.319319 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 02:15:06.319319 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 02:15:06.319319 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:06.319319 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:06.319319 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:06.319319 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:06.319319 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:06.319319 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:06.319319 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:06.319319 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:06.319319 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:06.319319 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:06.319319 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:06.319319 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:06.320122 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:06.320122 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:06.320122 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:06.320122 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:06.320122 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 02:15:06.320122 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 02:15:06.320122 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 02:15:06.320122 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 02:15:06.320122 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 02:15:06.320122 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 02:15:06.320122 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 02:15:06.320122 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:15:06.323387 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 186/385] 10.222.0.66 () {56 vars in 1114 bytes} [Mon Jul 18 02:15:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 02:15:06.330290 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14fc1e6-4940-410c-b327-a5346cbab180 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:06.342846 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2660fdf-bd2a-43a1-a7c7-5e31a2972a91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:06.344246 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2660fdf-bd2a-43a1-a7c7-5e31a2972a91 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:06.344808 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2660fdf-bd2a-43a1-a7c7-5e31a2972a91 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:06.345163 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2660fdf-bd2a-43a1-a7c7-5e31a2972a91 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:06.388175 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2660fdf-bd2a-43a1-a7c7-5e31a2972a91 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 02:15:06.404430 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-a2660fdf-bd2a-43a1-a7c7-5e31a2972a91 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 02:15:06.404430 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:06.404430 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:06.404430 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:06.404430 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:06.404430 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:06.404430 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:06.404430 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:06.404430 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:06.404430 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:06.404430 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:06.404430 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:06.404430 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 02:15:06.404430 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 02:15:06.404430 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 02:15:06.404430 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 02:15:06.405380 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 02:15:06.405380 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 02:15:06.405380 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 02:15:06.405380 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 02:15:06.405380 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:06.405380 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:06.405380 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:06.405380 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:06.405380 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:06.405380 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:06.405380 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:06.405380 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:06.405380 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:06.405380 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:06.405380 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:06.405380 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:06.406925 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:06.406925 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:06.406925 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:06.406925 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:06.406925 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 02:15:06.406925 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 02:15:06.406925 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 02:15:06.406925 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 02:15:06.406925 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 02:15:06.406925 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 02:15:06.406925 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 02:15:06.406925 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:15:06.406925 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 200/386] 10.222.0.66 () {56 vars in 1099 bytes} [Mon Jul 18 02:15:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 401 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:06.412633 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2660fdf-bd2a-43a1-a7c7-5e31a2972a91 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:06.447898 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14fc1e6-4940-410c-b327-a5346cbab180 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:06.472625 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:06.487026 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b4ebdf-7467-4bc7-8e30-3479ca3d700b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:06.488033 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b4ebdf-7467-4bc7-8e30-3479ca3d700b None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:06.488512 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b4ebdf-7467-4bc7-8e30-3479ca3d700b None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:06.488979 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b4ebdf-7467-4bc7-8e30-3479ca3d700b None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:06.489913 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b4ebdf-7467-4bc7-8e30-3479ca3d700b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:06.490498 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b4ebdf-7467-4bc7-8e30-3479ca3d700b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:06.492105 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b4ebdf-7467-4bc7-8e30-3479ca3d700b None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:06.502312 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 187/387] 10.222.0.66 () {56 vars in 1111 bytes} [Mon Jul 18 02:15:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 02:15:06.509202 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b4ebdf-7467-4bc7-8e30-3479ca3d700b None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:06.525174 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:06.577126 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2660fdf-bd2a-43a1-a7c7-5e31a2972a91 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:06.626878 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec44b97b-6572-4877-9db0-4a5cafbf5e83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:06.627573 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec44b97b-6572-4877-9db0-4a5cafbf5e83 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:06.627860 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec44b97b-6572-4877-9db0-4a5cafbf5e83 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:06.628138 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec44b97b-6572-4877-9db0-4a5cafbf5e83 None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:06.628904 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec44b97b-6572-4877-9db0-4a5cafbf5e83 None admin] RBAC: Adding query filter params (name=member) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:06.629212 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec44b97b-6572-4877-9db0-4a5cafbf5e83 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:06.630651 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec44b97b-6572-4877-9db0-4a5cafbf5e83 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:06.639339 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b4ebdf-7467-4bc7-8e30-3479ca3d700b None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:06.639850 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 201/388] 10.222.0.66 () {56 vars in 1101 bytes} [Mon Jul 18 02:15:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:06.645882 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec44b97b-6572-4877-9db0-4a5cafbf5e83 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:06.678375 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5fb362-490e-4105-ac37-9aad3c0d7669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:06.679251 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5fb362-490e-4105-ac37-9aad3c0d7669 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:06.679528 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5fb362-490e-4105-ac37-9aad3c0d7669 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:06.679796 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5fb362-490e-4105-ac37-9aad3c0d7669 None admin] PATH_INFO: `/v3/groups/02deab00a40e4493bb6ddac19fc25741` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:06.681042 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5fb362-490e-4105-ac37-9aad3c0d7669 None admin] RBAC: Authorizing `identity:get_group(group_id=02deab00a40e4493bb6ddac19fc25741)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:06.682558 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5fb362-490e-4105-ac37-9aad3c0d7669 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:06.684487 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 188/389] 10.222.0.66 () {56 vars in 1180 bytes} [Mon Jul 18 02:15:06 2022] GET /identity/v3/groups/02deab00a40e4493bb6ddac19fc25741 => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 02:15:06.690139 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5fb362-490e-4105-ac37-9aad3c0d7669 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:06.746587 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:06.797213 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5fb362-490e-4105-ac37-9aad3c0d7669 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:06.797523 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec44b97b-6572-4877-9db0-4a5cafbf5e83 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:06.833869 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edecf589-68de-432e-a52f-a041622774eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:06.834780 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edecf589-68de-432e-a52f-a041622774eb None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:06.835059 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edecf589-68de-432e-a52f-a041622774eb None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:06.835334 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edecf589-68de-432e-a52f-a041622774eb None admin] PATH_INFO: `/v3/projects/a5370165a2354ba6aec1bb43138f4838` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:06.836514 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edecf589-68de-432e-a52f-a041622774eb None admin] RBAC: Authorizing `identity:get_project(project_id=a5370165a2354ba6aec1bb43138f4838)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:06.838326 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edecf589-68de-432e-a52f-a041622774eb None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:06.840400 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 189/390] 10.222.0.66 () {56 vars in 1186 bytes} [Mon Jul 18 02:15:06 2022] GET /identity/v3/projects/a5370165a2354ba6aec1bb43138f4838 => generated 298 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 02:15:06.845722 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:06.846263 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edecf589-68de-432e-a52f-a041622774eb None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:06.858770 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d46a66-132b-4c19-b0f3-39222db893cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:06.861562 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d46a66-132b-4c19-b0f3-39222db893cb None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:06.861562 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d46a66-132b-4c19-b0f3-39222db893cb None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:06.861562 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d46a66-132b-4c19-b0f3-39222db893cb None admin] PATH_INFO: `/v3/groups/02deab00a40e4493bb6ddac19fc25741` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:06.861967 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d46a66-132b-4c19-b0f3-39222db893cb None admin] RBAC: Authorizing `identity:get_group(group_id=02deab00a40e4493bb6ddac19fc25741)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:06.863464 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d46a66-132b-4c19-b0f3-39222db893cb None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:06.865185 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 202/391] 10.222.0.66 () {56 vars in 1180 bytes} [Mon Jul 18 02:15:06 2022] GET /identity/v3/groups/02deab00a40e4493bb6ddac19fc25741 => generated 226 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 02:15:06.870666 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d46a66-132b-4c19-b0f3-39222db893cb None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:06.954533 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edecf589-68de-432e-a52f-a041622774eb None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:06.975565 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:06.990759 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f3890f-c412-484f-9931-f0aefcc11b03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:06.993458 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f3890f-c412-484f-9931-f0aefcc11b03 None admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:06.993458 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f3890f-c412-484f-9931-f0aefcc11b03 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:06.993458 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f3890f-c412-484f-9931-f0aefcc11b03 None admin] PATH_INFO: `/v3/projects/a5370165a2354ba6aec1bb43138f4838/groups/02deab00a40e4493bb6ddac19fc25741/roles/a5b895323e7b4f0db2491abe9ad57ee9` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:06.995238 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f3890f-c412-484f-9931-f0aefcc11b03 None admin] RBAC: Authorizing `identity:create_grant(project_id=a5370165a2354ba6aec1bb43138f4838, group_id=02deab00a40e4493bb6ddac19fc25741, role_id=a5b895323e7b4f0db2491abe9ad57ee9)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:06.996881 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f3890f-c412-484f-9931-f0aefcc11b03 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:07.021174 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d46a66-132b-4c19-b0f3-39222db893cb None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:07.066454 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b0b9d1-33a2-42b3-9ba8-4a3e5c3216e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:07.067326 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b0b9d1-33a2-42b3-9ba8-4a3e5c3216e2 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:07.067616 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b0b9d1-33a2-42b3-9ba8-4a3e5c3216e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:07.067891 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b0b9d1-33a2-42b3-9ba8-4a3e5c3216e2 None admin] PATH_INFO: `/v3/projects/a5370165a2354ba6aec1bb43138f4838` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:07.069114 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b0b9d1-33a2-42b3-9ba8-4a3e5c3216e2 None admin] RBAC: Authorizing `identity:get_project(project_id=a5370165a2354ba6aec1bb43138f4838)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:07.070710 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b0b9d1-33a2-42b3-9ba8-4a3e5c3216e2 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:07.072620 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 203/392] 10.222.0.66 () {56 vars in 1186 bytes} [Mon Jul 18 02:15:06 2022] GET /identity/v3/projects/a5370165a2354ba6aec1bb43138f4838 => generated 298 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 02:15:07.079098 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b0b9d1-33a2-42b3-9ba8-4a3e5c3216e2 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:07.101286 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 190/393] 10.222.0.66 () {58 vars in 1442 bytes} [Mon Jul 18 02:15:06 2022] PUT /identity/v3/projects/a5370165a2354ba6aec1bb43138f4838/groups/02deab00a40e4493bb6ddac19fc25741/roles/a5b895323e7b4f0db2491abe9ad57ee9 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:15:07.157475 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:07.195620 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b0b9d1-33a2-42b3-9ba8-4a3e5c3216e2 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:07.232692 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d0e04a-bf78-4999-9807-53eecac74990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:07.233503 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d0e04a-bf78-4999-9807-53eecac74990 None admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:07.233788 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d0e04a-bf78-4999-9807-53eecac74990 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:07.234155 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d0e04a-bf78-4999-9807-53eecac74990 None admin] PATH_INFO: `/v3/projects/a5370165a2354ba6aec1bb43138f4838/groups/02deab00a40e4493bb6ddac19fc25741/roles/65feabf3b27944a8b01a86c8ac608457` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:07.236259 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d0e04a-bf78-4999-9807-53eecac74990 None admin] RBAC: Authorizing `identity:create_grant(project_id=a5370165a2354ba6aec1bb43138f4838, group_id=02deab00a40e4493bb6ddac19fc25741, role_id=65feabf3b27944a8b01a86c8ac608457)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:07.237775 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d0e04a-bf78-4999-9807-53eecac74990 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:07.335568 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 204/394] 10.222.0.66 () {58 vars in 1442 bytes} [Mon Jul 18 02:15:07 2022] PUT /identity/v3/projects/a5370165a2354ba6aec1bb43138f4838/groups/02deab00a40e4493bb6ddac19fc25741/roles/65feabf3b27944a8b01a86c8ac608457 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:15:08.161436 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f15ff6-b341-434d-81b5-c82dff740792 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:08.162058 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f15ff6-b341-434d-81b5-c82dff740792 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:08.162058 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f15ff6-b341-434d-81b5-c82dff740792 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:08.163669 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 191/395] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:15:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:15:08.169301 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc9a875-3715-4246-b102-96616f22dc3a None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:08.169776 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc9a875-3715-4246-b102-96616f22dc3a None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:08.169860 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc9a875-3715-4246-b102-96616f22dc3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:08.188682 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-7bc9a875-3715-4246-b102-96616f22dc3a None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:15:08.189889 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-7bc9a875-3715-4246-b102-96616f22dc3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:15:08.260095 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 205/396] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:15:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 02:15:08.282770 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f15ff6-b341-434d-81b5-c82dff740792 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:08.292214 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd5840b-077c-430c-b0e8-7a09dbe6a875 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:08.293127 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd5840b-077c-430c-b0e8-7a09dbe6a875 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:08.293416 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd5840b-077c-430c-b0e8-7a09dbe6a875 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:08.295105 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 206/397] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:15:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:15:08.303178 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3819ddcd-667e-4c8f-8397-a204ff10a791 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:08.303474 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3819ddcd-667e-4c8f-8397-a204ff10a791 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:08.303753 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3819ddcd-667e-4c8f-8397-a204ff10a791 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:08.332306 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-3819ddcd-667e-4c8f-8397-a204ff10a791 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:15:08.333510 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-3819ddcd-667e-4c8f-8397-a204ff10a791 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:15:08.449800 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:08.465131 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f15ff6-b341-434d-81b5-c82dff740792 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:08.485356 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 207/398] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:15:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 02:15:08.511209 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fe322e-9b39-4227-bece-a69790d53051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:08.513008 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fe322e-9b39-4227-bece-a69790d53051 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:08.513008 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fe322e-9b39-4227-bece-a69790d53051 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:08.513008 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fe322e-9b39-4227-bece-a69790d53051 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:08.524973 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fe322e-9b39-4227-bece-a69790d53051 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 02:15:08.533849 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-e2fe322e-9b39-4227-bece-a69790d53051 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 02:15:08.533849 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:08.533849 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:08.533849 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:08.533849 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:08.533849 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:08.533849 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:08.533849 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:08.533849 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:08.533849 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:08.533849 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:08.533849 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:08.533849 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 02:15:08.533849 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 02:15:08.533849 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 02:15:08.533849 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 02:15:08.534854 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 02:15:08.534854 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 02:15:08.534854 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 02:15:08.534854 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 02:15:08.534854 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:08.534854 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:08.534854 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:08.534854 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:08.534854 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:08.534854 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:08.534854 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:08.534854 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:08.534854 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:08.534854 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:08.534854 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:08.534854 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:08.535696 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:08.535696 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:08.535696 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:08.535696 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:08.535696 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 02:15:08.535696 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 02:15:08.535696 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 02:15:08.535696 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 02:15:08.535696 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 02:15:08.535696 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 02:15:08.535696 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 02:15:08.535696 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:15:08.536340 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3819ddcd-667e-4c8f-8397-a204ff10a791 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:08.538273 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 192/399] 10.222.0.66 () {56 vars in 1096 bytes} [Mon Jul 18 02:15:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:08.548094 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fe322e-9b39-4227-bece-a69790d53051 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:08.627277 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:08.715169 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3819ddcd-667e-4c8f-8397-a204ff10a791 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:08.721942 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fe322e-9b39-4227-bece-a69790d53051 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:08.761648 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6226030c-a81c-4b23-b9ed-59d78f4a363d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:08.762369 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6226030c-a81c-4b23-b9ed-59d78f4a363d None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:08.762655 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6226030c-a81c-4b23-b9ed-59d78f4a363d None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:08.762928 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6226030c-a81c-4b23-b9ed-59d78f4a363d None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:08.763680 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6226030c-a81c-4b23-b9ed-59d78f4a363d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:08.763989 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6226030c-a81c-4b23-b9ed-59d78f4a363d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:08.765511 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6226030c-a81c-4b23-b9ed-59d78f4a363d None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:08.774993 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 193/400] 10.222.0.66 () {56 vars in 1099 bytes} [Mon Jul 18 02:15:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 02:15:08.780183 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6226030c-a81c-4b23-b9ed-59d78f4a363d None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:08.801771 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cf2a23-ed49-426f-895b-30ef737efbcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:08.801771 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cf2a23-ed49-426f-895b-30ef737efbcf None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:08.802058 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cf2a23-ed49-426f-895b-30ef737efbcf None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:08.802316 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cf2a23-ed49-426f-895b-30ef737efbcf None admin] PATH_INFO: `/v3/roles/member` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:08.822143 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cf2a23-ed49-426f-895b-30ef737efbcf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 02:15:08.831788 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-b3cf2a23-ed49-426f-895b-30ef737efbcf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 02:15:08.831788 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:08.831788 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:08.831788 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:08.831788 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:08.831788 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:08.831788 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:08.831788 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:08.831788 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:08.831788 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:08.831788 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:08.831788 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:08.831788 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 02:15:08.831788 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 02:15:08.831788 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 02:15:08.831788 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 02:15:08.832700 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 02:15:08.832700 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 02:15:08.832700 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 02:15:08.832700 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 02:15:08.832700 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:08.832700 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:08.832700 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:08.832700 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:08.832700 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:08.832700 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:08.832700 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:08.832700 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:08.832700 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:08.832700 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:08.832700 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:08.832700 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:08.833534 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:08.833534 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:08.833534 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:08.833534 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:08.833534 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 02:15:08.833534 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 02:15:08.833534 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 02:15:08.833534 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 02:15:08.833534 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 02:15:08.833534 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 02:15:08.833534 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 02:15:08.833534 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:15:08.833534 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 208/401] 10.222.0.66 () {56 vars in 1099 bytes} [Mon Jul 18 02:15:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 02:15:08.834797 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1a1edd-2c9a-453c-832d-8182cb4f90bf None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:08.834907 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1a1edd-2c9a-453c-832d-8182cb4f90bf None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:08.835144 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1a1edd-2c9a-453c-832d-8182cb4f90bf None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:08.836820 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 209/402] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:15:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:15:08.839188 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1a1edd-2c9a-453c-832d-8182cb4f90bf None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:08.897945 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:08.915878 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6226030c-a81c-4b23-b9ed-59d78f4a363d None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:08.917443 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:08.951856 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4f6fb1-3d66-4e47-a1fe-fc392b0f95f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:08.953266 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4f6fb1-3d66-4e47-a1fe-fc392b0f95f4 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:08.953549 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4f6fb1-3d66-4e47-a1fe-fc392b0f95f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:08.953820 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4f6fb1-3d66-4e47-a1fe-fc392b0f95f4 None admin] PATH_INFO: `/v3/groups/e429b16d0a744db79625f143ae9e4696` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:08.955110 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4f6fb1-3d66-4e47-a1fe-fc392b0f95f4 None admin] RBAC: Authorizing `identity:get_group(group_id=e429b16d0a744db79625f143ae9e4696)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:08.956648 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4f6fb1-3d66-4e47-a1fe-fc392b0f95f4 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:08.959190 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1a1edd-2c9a-453c-832d-8182cb4f90bf None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:08.959500 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 194/403] 10.222.0.66 () {56 vars in 1180 bytes} [Mon Jul 18 02:15:08 2022] GET /identity/v3/groups/e429b16d0a744db79625f143ae9e4696 => generated 229 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:08.960855 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4069ba4-2c66-4ce9-94c4-e95a5b363a35 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:08.961134 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4069ba4-2c66-4ce9-94c4-e95a5b363a35 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:08.961455 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4069ba4-2c66-4ce9-94c4-e95a5b363a35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:08.981401 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-f4069ba4-2c66-4ce9-94c4-e95a5b363a35 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:15:08.990124 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-f4069ba4-2c66-4ce9-94c4-e95a5b363a35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:15:09.004979 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08621bb-f28a-43bc-ab0f-f8173eb70861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:09.005654 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08621bb-f28a-43bc-ab0f-f8173eb70861 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:09.005939 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08621bb-f28a-43bc-ab0f-f8173eb70861 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:09.006281 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08621bb-f28a-43bc-ab0f-f8173eb70861 None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:09.007147 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08621bb-f28a-43bc-ab0f-f8173eb70861 None admin] RBAC: Adding query filter params (name=member) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:09.007561 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08621bb-f28a-43bc-ab0f-f8173eb70861 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:09.008957 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08621bb-f28a-43bc-ab0f-f8173eb70861 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:09.019923 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 210/404] 10.222.0.66 () {56 vars in 1101 bytes} [Mon Jul 18 02:15:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:09.020883 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08621bb-f28a-43bc-ab0f-f8173eb70861 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:09.056546 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 195/405] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:15:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 02:15:09.057210 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4069ba4-2c66-4ce9-94c4-e95a5b363a35 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:09.157139 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08621bb-f28a-43bc-ab0f-f8173eb70861 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:09.159337 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:09.162527 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4069ba4-2c66-4ce9-94c4-e95a5b363a35 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:09.168356 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:09.195164 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6de049-1caf-41eb-9f42-d8c81a1c2991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:09.196081 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6de049-1caf-41eb-9f42-d8c81a1c2991 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:09.196371 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6de049-1caf-41eb-9f42-d8c81a1c2991 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:09.196646 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6de049-1caf-41eb-9f42-d8c81a1c2991 None admin] PATH_INFO: `/v3/projects/a05e40a15656441b94f5763e4d68d305` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:09.197876 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6de049-1caf-41eb-9f42-d8c81a1c2991 None admin] RBAC: Authorizing `identity:get_project(project_id=a05e40a15656441b94f5763e4d68d305)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:09.199605 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6de049-1caf-41eb-9f42-d8c81a1c2991 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:09.201330 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 211/406] 10.222.0.66 () {56 vars in 1186 bytes} [Mon Jul 18 02:15:09 2022] GET /identity/v3/projects/a05e40a15656441b94f5763e4d68d305 => generated 344 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:09.201624 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777a51cb-63f5-40e7-bcb2-86fd25a303ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:09.202118 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6de049-1caf-41eb-9f42-d8c81a1c2991 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:09.202549 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777a51cb-63f5-40e7-bcb2-86fd25a303ad None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:09.203835 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777a51cb-63f5-40e7-bcb2-86fd25a303ad None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:09.203835 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777a51cb-63f5-40e7-bcb2-86fd25a303ad None admin] PATH_INFO: `/v3/groups/02deab00a40e4493bb6ddac19fc25741` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:09.205358 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777a51cb-63f5-40e7-bcb2-86fd25a303ad None admin] RBAC: Authorizing `identity:get_group(group_id=02deab00a40e4493bb6ddac19fc25741)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:09.206913 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777a51cb-63f5-40e7-bcb2-86fd25a303ad None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:09.208695 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 196/407] 10.222.0.66 () {56 vars in 1180 bytes} [Mon Jul 18 02:15:09 2022] GET /identity/v3/groups/02deab00a40e4493bb6ddac19fc25741 => generated 226 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:09.209456 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777a51cb-63f5-40e7-bcb2-86fd25a303ad None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:09.311071 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6de049-1caf-41eb-9f42-d8c81a1c2991 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:09.317921 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777a51cb-63f5-40e7-bcb2-86fd25a303ad None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:09.356169 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcbef42-acf8-4fa9-9ff3-6224987effe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:09.357059 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcbef42-acf8-4fa9-9ff3-6224987effe2 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:09.357459 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcbef42-acf8-4fa9-9ff3-6224987effe2 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:09.357794 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcbef42-acf8-4fa9-9ff3-6224987effe2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:09.359876 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4146a300-b998-4227-9844-dafe435ba165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:09.360888 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4146a300-b998-4227-9844-dafe435ba165 None admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:09.361268 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4146a300-b998-4227-9844-dafe435ba165 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:09.361642 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4146a300-b998-4227-9844-dafe435ba165 None admin] PATH_INFO: `/v3/projects/a05e40a15656441b94f5763e4d68d305/groups/e429b16d0a744db79625f143ae9e4696/roles/d4aa3324f0c84887b41d0dc4acf7f30e` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:09.364810 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4146a300-b998-4227-9844-dafe435ba165 None admin] RBAC: Authorizing `identity:create_grant(project_id=a05e40a15656441b94f5763e4d68d305, group_id=e429b16d0a744db79625f143ae9e4696, role_id=d4aa3324f0c84887b41d0dc4acf7f30e)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:09.366740 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4146a300-b998-4227-9844-dafe435ba165 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:09.373231 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcbef42-acf8-4fa9-9ff3-6224987effe2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 02:15:09.382560 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-efcbef42-acf8-4fa9-9ff3-6224987effe2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 02:15:09.382560 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:09.382560 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:09.382560 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:09.382560 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:09.382560 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:09.382560 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:09.382560 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:09.382560 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:09.382560 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:09.382560 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:09.382560 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:09.382560 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 02:15:09.382560 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 02:15:09.382560 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 02:15:09.382560 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 02:15:09.383452 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 02:15:09.383452 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 02:15:09.383452 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 02:15:09.383452 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 02:15:09.383452 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:09.383452 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:09.383452 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:09.383452 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:09.383452 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:09.383452 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:09.383452 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:09.383452 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:09.383452 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:09.383452 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:09.383452 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:09.383452 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:09.384582 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:09.384582 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:09.384582 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:09.384582 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:09.384582 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 02:15:09.384582 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 02:15:09.384582 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 02:15:09.384582 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 02:15:09.384582 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 02:15:09.384582 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 02:15:09.384582 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 02:15:09.384582 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:15:09.384582 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 212/408] 10.222.0.66 () {56 vars in 1114 bytes} [Mon Jul 18 02:15:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:09.384582 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcbef42-acf8-4fa9-9ff3-6224987effe2 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:09.428960 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:09.453151 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 197/409] 10.222.0.66 () {58 vars in 1442 bytes} [Mon Jul 18 02:15:09 2022] PUT /identity/v3/projects/a05e40a15656441b94f5763e4d68d305/groups/e429b16d0a744db79625f143ae9e4696/roles/d4aa3324f0c84887b41d0dc4acf7f30e => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:15:09.454583 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4146a300-b998-4227-9844-dafe435ba165 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:09.518519 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:09.534425 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcbef42-acf8-4fa9-9ff3-6224987effe2 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:09.609341 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4146a300-b998-4227-9844-dafe435ba165 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:09.629699 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0a73b8-ca6c-4dcc-bcfb-1fc1acec3c30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:09.630668 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0a73b8-ca6c-4dcc-bcfb-1fc1acec3c30 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:09.634550 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0a73b8-ca6c-4dcc-bcfb-1fc1acec3c30 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:09.634550 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0a73b8-ca6c-4dcc-bcfb-1fc1acec3c30 None admin] PATH_INFO: `/v3/projects/9748772803a44ccea6827637294a3a97` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:09.634550 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0a73b8-ca6c-4dcc-bcfb-1fc1acec3c30 None admin] RBAC: Authorizing `identity:get_project(project_id=9748772803a44ccea6827637294a3a97)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:09.634550 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0a73b8-ca6c-4dcc-bcfb-1fc1acec3c30 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:09.636400 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 213/410] 10.222.0.66 () {56 vars in 1186 bytes} [Mon Jul 18 02:15:09 2022] GET /identity/v3/projects/9748772803a44ccea6827637294a3a97 => generated 302 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:09.642404 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0a73b8-ca6c-4dcc-bcfb-1fc1acec3c30 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:09.652163 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3a91cc-acf6-49e0-8dba-7ae6987b65e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:09.653725 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3a91cc-acf6-49e0-8dba-7ae6987b65e5 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:09.654262 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3a91cc-acf6-49e0-8dba-7ae6987b65e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:09.654729 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3a91cc-acf6-49e0-8dba-7ae6987b65e5 None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:09.656116 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3a91cc-acf6-49e0-8dba-7ae6987b65e5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:09.657008 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3a91cc-acf6-49e0-8dba-7ae6987b65e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:09.659105 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3a91cc-acf6-49e0-8dba-7ae6987b65e5 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:09.669424 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 198/411] 10.222.0.66 () {56 vars in 1111 bytes} [Mon Jul 18 02:15:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:09.676639 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3a91cc-acf6-49e0-8dba-7ae6987b65e5 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:09.763883 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0a73b8-ca6c-4dcc-bcfb-1fc1acec3c30 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:09.767116 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:09.801170 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555761f7-23cd-40b7-932e-e58d7c98073d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:09.802149 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555761f7-23cd-40b7-932e-e58d7c98073d None admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:09.802447 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555761f7-23cd-40b7-932e-e58d7c98073d None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:09.802726 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555761f7-23cd-40b7-932e-e58d7c98073d None admin] PATH_INFO: `/v3/projects/9748772803a44ccea6827637294a3a97/groups/02deab00a40e4493bb6ddac19fc25741/roles/65feabf3b27944a8b01a86c8ac608457` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:09.804736 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555761f7-23cd-40b7-932e-e58d7c98073d None admin] RBAC: Authorizing `identity:create_grant(project_id=9748772803a44ccea6827637294a3a97, group_id=02deab00a40e4493bb6ddac19fc25741, role_id=65feabf3b27944a8b01a86c8ac608457)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:09.806209 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555761f7-23cd-40b7-932e-e58d7c98073d None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:09.810931 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:09.814851 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3a91cc-acf6-49e0-8dba-7ae6987b65e5 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:09.850546 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7af4ce-d406-4465-9f48-2fb56204b326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:09.851413 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7af4ce-d406-4465-9f48-2fb56204b326 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:09.851699 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7af4ce-d406-4465-9f48-2fb56204b326 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:09.851975 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7af4ce-d406-4465-9f48-2fb56204b326 None admin] PATH_INFO: `/v3/groups/02deab00a40e4493bb6ddac19fc25741` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:09.853221 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7af4ce-d406-4465-9f48-2fb56204b326 None admin] RBAC: Authorizing `identity:get_group(group_id=02deab00a40e4493bb6ddac19fc25741)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:09.854751 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7af4ce-d406-4465-9f48-2fb56204b326 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:09.856558 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 199/412] 10.222.0.66 () {56 vars in 1180 bytes} [Mon Jul 18 02:15:09 2022] GET /identity/v3/groups/02deab00a40e4493bb6ddac19fc25741 => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 02:15:09.861428 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c00cc97-560c-48ab-bddf-0fd6137febd6 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:09.861428 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c00cc97-560c-48ab-bddf-0fd6137febd6 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:09.861428 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c00cc97-560c-48ab-bddf-0fd6137febd6 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:09.862159 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 200/413] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:15:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:15:09.863346 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c00cc97-560c-48ab-bddf-0fd6137febd6 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:09.879349 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 214/414] 10.222.0.66 () {58 vars in 1442 bytes} [Mon Jul 18 02:15:09 2022] PUT /identity/v3/projects/9748772803a44ccea6827637294a3a97/groups/02deab00a40e4493bb6ddac19fc25741/roles/65feabf3b27944a8b01a86c8ac608457 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 02:15:09.886785 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7438f52d-891a-43bf-8bd3-27ef3217b01e None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:09.887500 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7438f52d-891a-43bf-8bd3-27ef3217b01e None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:09.887989 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7438f52d-891a-43bf-8bd3-27ef3217b01e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:09.924869 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-7438f52d-891a-43bf-8bd3-27ef3217b01e None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:15:09.929461 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-7438f52d-891a-43bf-8bd3-27ef3217b01e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:15:10.062636 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c00cc97-560c-48ab-bddf-0fd6137febd6 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:10.067366 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 215/415] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:15:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 02:15:10.102551 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884a30fb-fcd3-408e-b3f2-4d8921529119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:10.103763 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884a30fb-fcd3-408e-b3f2-4d8921529119 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:10.104259 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884a30fb-fcd3-408e-b3f2-4d8921529119 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:10.104736 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884a30fb-fcd3-408e-b3f2-4d8921529119 None admin] PATH_INFO: `/v3/projects/9748772803a44ccea6827637294a3a97` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:10.106119 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884a30fb-fcd3-408e-b3f2-4d8921529119 None admin] RBAC: Authorizing `identity:get_project(project_id=9748772803a44ccea6827637294a3a97)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:10.107818 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884a30fb-fcd3-408e-b3f2-4d8921529119 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:10.110515 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7438f52d-891a-43bf-8bd3-27ef3217b01e None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:10.110661 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 201/416] 10.222.0.66 () {56 vars in 1186 bytes} [Mon Jul 18 02:15:09 2022] GET /identity/v3/projects/9748772803a44ccea6827637294a3a97 => generated 302 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:10.118659 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884a30fb-fcd3-408e-b3f2-4d8921529119 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:10.150467 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:10.233514 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:10.239072 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884a30fb-fcd3-408e-b3f2-4d8921529119 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:10.241817 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7438f52d-891a-43bf-8bd3-27ef3217b01e None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:10.278725 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a6f248-d9f5-4c5e-bcd7-96dcf3bf907f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:10.279981 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a6f248-d9f5-4c5e-bcd7-96dcf3bf907f None admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:10.280491 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a6f248-d9f5-4c5e-bcd7-96dcf3bf907f None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:10.281577 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a6f248-d9f5-4c5e-bcd7-96dcf3bf907f None admin] PATH_INFO: `/v3/projects/9748772803a44ccea6827637294a3a97/groups/02deab00a40e4493bb6ddac19fc25741/roles/a5b895323e7b4f0db2491abe9ad57ee9` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:10.284230 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a6f248-d9f5-4c5e-bcd7-96dcf3bf907f None admin] RBAC: Authorizing `identity:create_grant(project_id=9748772803a44ccea6827637294a3a97, group_id=02deab00a40e4493bb6ddac19fc25741, role_id=a5b895323e7b4f0db2491abe9ad57ee9)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:10.295068 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a6f248-d9f5-4c5e-bcd7-96dcf3bf907f None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:10.295213 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efcad21-08fe-4492-8d42-5bb5be4b607d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:10.295213 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efcad21-08fe-4492-8d42-5bb5be4b607d None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:10.295213 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efcad21-08fe-4492-8d42-5bb5be4b607d None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:10.295213 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efcad21-08fe-4492-8d42-5bb5be4b607d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:10.301699 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efcad21-08fe-4492-8d42-5bb5be4b607d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 02:15:10.310001 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-4efcad21-08fe-4492-8d42-5bb5be4b607d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 02:15:10.310001 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:10.310001 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:10.310001 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:10.310001 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:10.310001 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:10.310001 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:10.310001 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:10.310001 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:10.310001 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:10.310001 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:10.310001 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:10.310001 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 02:15:10.310001 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 02:15:10.310001 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 02:15:10.310001 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 02:15:10.310918 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 02:15:10.310918 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 02:15:10.310918 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 02:15:10.310918 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 02:15:10.310918 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:10.310918 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:10.310918 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:10.310918 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:10.310918 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:10.310918 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:10.310918 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:10.310918 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:10.310918 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:10.310918 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:10.310918 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:10.310918 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:10.311742 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:10.311742 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:10.311742 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:10.311742 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:10.311742 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 02:15:10.311742 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 02:15:10.311742 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 02:15:10.311742 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 02:15:10.311742 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 02:15:10.311742 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 02:15:10.311742 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 02:15:10.311742 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:15:10.311742 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 216/417] 10.222.0.66 () {56 vars in 1114 bytes} [Mon Jul 18 02:15:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:10.313500 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1f517f-32bd-45c1-a64f-14771f8d77f6 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:10.313729 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1f517f-32bd-45c1-a64f-14771f8d77f6 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:10.314543 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1f517f-32bd-45c1-a64f-14771f8d77f6 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:10.322144 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 217/418] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:15:10 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:15:10.322144 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f0519d-5930-451b-8278-2327986f9bf9 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:10.322339 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f0519d-5930-451b-8278-2327986f9bf9 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:10.322605 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f0519d-5930-451b-8278-2327986f9bf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:10.339338 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-55f0519d-5930-451b-8278-2327986f9bf9 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:15:10.340543 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-55f0519d-5930-451b-8278-2327986f9bf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:15:10.386822 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 202/419] 10.222.0.66 () {58 vars in 1442 bytes} [Mon Jul 18 02:15:10 2022] PUT /identity/v3/projects/9748772803a44ccea6827637294a3a97/groups/02deab00a40e4493bb6ddac19fc25741/roles/a5b895323e7b4f0db2491abe9ad57ee9 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 02:15:10.388698 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a6f248-d9f5-4c5e-bcd7-96dcf3bf907f None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:10.419374 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 218/420] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:15:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 02:15:10.505119 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f0519d-5930-451b-8278-2327986f9bf9 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:10.543425 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a6f248-d9f5-4c5e-bcd7-96dcf3bf907f None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:10.586869 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebc404b-9c41-42ba-aa98-6237422c99b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:10.587929 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebc404b-9c41-42ba-aa98-6237422c99b6 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:10.588364 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebc404b-9c41-42ba-aa98-6237422c99b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:10.589018 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebc404b-9c41-42ba-aa98-6237422c99b6 None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:10.589817 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebc404b-9c41-42ba-aa98-6237422c99b6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:10.590172 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebc404b-9c41-42ba-aa98-6237422c99b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:10.591568 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebc404b-9c41-42ba-aa98-6237422c99b6 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:10.594303 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:10.612155 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 203/421] 10.222.0.66 () {56 vars in 1111 bytes} [Mon Jul 18 02:15:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:10.617750 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebc404b-9c41-42ba-aa98-6237422c99b6 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:10.643131 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:10.669847 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f0519d-5930-451b-8278-2327986f9bf9 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:10.741711 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d5c108-e146-4bb6-82a8-edc599b36834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:10.742836 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d5c108-e146-4bb6-82a8-edc599b36834 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:10.743121 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d5c108-e146-4bb6-82a8-edc599b36834 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:10.743394 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d5c108-e146-4bb6-82a8-edc599b36834 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:10.766306 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d5c108-e146-4bb6-82a8-edc599b36834 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 02:15:10.774284 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-d5d5c108-e146-4bb6-82a8-edc599b36834 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 02:15:10.774284 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:10.774284 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:10.774284 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:10.774284 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:10.774284 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:10.774284 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:10.774284 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:10.774284 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:10.774284 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:10.774284 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:10.774284 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:10.774284 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 02:15:10.774284 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 02:15:10.774284 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 02:15:10.774284 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 02:15:10.775204 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 02:15:10.775204 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 02:15:10.775204 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 02:15:10.775204 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 02:15:10.775204 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:10.775204 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:10.775204 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:10.775204 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:10.775204 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:10.775204 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:10.775204 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:10.775204 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:10.775204 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:10.775204 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:10.775204 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:10.775204 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:10.776516 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:10.776516 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:10.776516 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:10.776516 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:10.776516 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 02:15:10.776516 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 02:15:10.776516 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 02:15:10.776516 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 02:15:10.776516 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 02:15:10.776516 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 02:15:10.776516 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 02:15:10.776516 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:15:10.776516 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 219/422] 10.222.0.66 () {56 vars in 1099 bytes} [Mon Jul 18 02:15:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:10.782654 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d5c108-e146-4bb6-82a8-edc599b36834 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:10.806467 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebc404b-9c41-42ba-aa98-6237422c99b6 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:10.867982 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b35cf0-8261-4a8a-abf9-a15b2733bec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:10.868645 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b35cf0-8261-4a8a-abf9-a15b2733bec7 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:10.868925 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b35cf0-8261-4a8a-abf9-a15b2733bec7 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:10.869195 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b35cf0-8261-4a8a-abf9-a15b2733bec7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:10.892525 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d5c108-e146-4bb6-82a8-edc599b36834 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:10.908743 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:10.930044 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2d3ece-6c21-4744-b9ad-23ac4ec59d83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:10.931016 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2d3ece-6c21-4744-b9ad-23ac4ec59d83 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:10.931476 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2d3ece-6c21-4744-b9ad-23ac4ec59d83 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:10.931932 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2d3ece-6c21-4744-b9ad-23ac4ec59d83 None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:10.932913 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2d3ece-6c21-4744-b9ad-23ac4ec59d83 None admin] RBAC: Adding query filter params (name=member) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:10.933426 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2d3ece-6c21-4744-b9ad-23ac4ec59d83 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:10.935088 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2d3ece-6c21-4744-b9ad-23ac4ec59d83 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:10.945264 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 220/423] 10.222.0.66 () {56 vars in 1101 bytes} [Mon Jul 18 02:15:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:10.950648 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2d3ece-6c21-4744-b9ad-23ac4ec59d83 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:10.961556 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:11.055166 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b35cf0-8261-4a8a-abf9-a15b2733bec7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:11.056777 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b35cf0-8261-4a8a-abf9-a15b2733bec7 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:11.065896 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2d3ece-6c21-4744-b9ad-23ac4ec59d83 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:11.104557 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d476adc9-2134-4ab3-9ddc-5a8cd47fcf22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:11.105506 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d476adc9-2134-4ab3-9ddc-5a8cd47fcf22 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:11.106274 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d476adc9-2134-4ab3-9ddc-5a8cd47fcf22 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:11.106733 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d476adc9-2134-4ab3-9ddc-5a8cd47fcf22 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:11.202158 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d476adc9-2134-4ab3-9ddc-5a8cd47fcf22 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:11.204390 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d476adc9-2134-4ab3-9ddc-5a8cd47fcf22 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:11.217688 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:11.291280 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 204/424] 10.222.0.66 () {58 vars in 1282 bytes} [Mon Jul 18 02:15:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 674 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 18 02:15:11.297392 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b35cf0-8261-4a8a-abf9-a15b2733bec7 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:11.402448 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:11.404381 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 221/425] 10.222.0.66 () {58 vars in 1282 bytes} [Mon Jul 18 02:15:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 02:15:11.413059 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d476adc9-2134-4ab3-9ddc-5a8cd47fcf22 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:11.517175 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d476adc9-2134-4ab3-9ddc-5a8cd47fcf22 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:11.524594 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:11.548668 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b35cf0-8261-4a8a-abf9-a15b2733bec7 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:11.554230 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e442c3e2-7877-4730-a2b3-829048b9db3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:11.554947 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e442c3e2-7877-4730-a2b3-829048b9db3b None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:11.555311 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e442c3e2-7877-4730-a2b3-829048b9db3b None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:11.555486 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e442c3e2-7877-4730-a2b3-829048b9db3b None admin] PATH_INFO: `/v3/projects/a5370165a2354ba6aec1bb43138f4838` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:11.556674 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e442c3e2-7877-4730-a2b3-829048b9db3b None admin] RBAC: Authorizing `identity:get_project(project_id=a5370165a2354ba6aec1bb43138f4838)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:11.558255 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e442c3e2-7877-4730-a2b3-829048b9db3b None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:11.560605 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 222/426] 10.222.0.66 () {56 vars in 1186 bytes} [Mon Jul 18 02:15:11 2022] GET /identity/v3/projects/a5370165a2354ba6aec1bb43138f4838 => generated 298 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:11.571558 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e442c3e2-7877-4730-a2b3-829048b9db3b None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:11.615039 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b1f0ef-30a2-49e3-b798-befdbb44d851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:11.615913 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b1f0ef-30a2-49e3-b798-befdbb44d851 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:11.616196 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b1f0ef-30a2-49e3-b798-befdbb44d851 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:11.616472 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b1f0ef-30a2-49e3-b798-befdbb44d851 None admin] PATH_INFO: `/v3/projects/a5370165a2354ba6aec1bb43138f4838` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:11.618304 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b1f0ef-30a2-49e3-b798-befdbb44d851 None admin] RBAC: Authorizing `identity:get_project(project_id=a5370165a2354ba6aec1bb43138f4838)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:11.619761 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b1f0ef-30a2-49e3-b798-befdbb44d851 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:11.623239 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 205/427] 10.222.0.66 () {56 vars in 1186 bytes} [Mon Jul 18 02:15:11 2022] GET /identity/v3/projects/a5370165a2354ba6aec1bb43138f4838 => generated 298 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:11.627724 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b1f0ef-30a2-49e3-b798-befdbb44d851 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:11.714526 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e442c3e2-7877-4730-a2b3-829048b9db3b None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:11.752862 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557bf076-661a-4400-8979-1c12f3e3ed8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:11.753986 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557bf076-661a-4400-8979-1c12f3e3ed8a None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:11.754211 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557bf076-661a-4400-8979-1c12f3e3ed8a None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:11.754493 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557bf076-661a-4400-8979-1c12f3e3ed8a None admin] PATH_INFO: `/v3/groups/02deab00a40e4493bb6ddac19fc25741` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:11.756337 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557bf076-661a-4400-8979-1c12f3e3ed8a None admin] RBAC: Authorizing `identity:get_group(group_id=02deab00a40e4493bb6ddac19fc25741)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:11.758360 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557bf076-661a-4400-8979-1c12f3e3ed8a None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:11.760538 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b1f0ef-30a2-49e3-b798-befdbb44d851 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:11.762281 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 223/428] 10.222.0.66 () {56 vars in 1180 bytes} [Mon Jul 18 02:15:11 2022] GET /identity/v3/groups/02deab00a40e4493bb6ddac19fc25741 => generated 226 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:11.763532 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9cbb01-c1c5-435e-a491-fd88380a19f9 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:11.763887 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9cbb01-c1c5-435e-a491-fd88380a19f9 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:11.764231 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9cbb01-c1c5-435e-a491-fd88380a19f9 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:11.765755 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 224/429] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:15:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:15:11.771273 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9cbb01-c1c5-435e-a491-fd88380a19f9 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:11.787491 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:11.815089 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c91e5fb-d155-44ca-9ae1-30012b0b9e2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:11.816901 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c91e5fb-d155-44ca-9ae1-30012b0b9e2e None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:11.817419 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c91e5fb-d155-44ca-9ae1-30012b0b9e2e None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:11.818023 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c91e5fb-d155-44ca-9ae1-30012b0b9e2e None admin] PATH_INFO: `/v3/groups/02deab00a40e4493bb6ddac19fc25741` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:11.820339 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c91e5fb-d155-44ca-9ae1-30012b0b9e2e None admin] RBAC: Authorizing `identity:get_group(group_id=02deab00a40e4493bb6ddac19fc25741)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:11.822015 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c91e5fb-d155-44ca-9ae1-30012b0b9e2e None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:11.825270 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 206/430] 10.222.0.66 () {56 vars in 1180 bytes} [Mon Jul 18 02:15:11 2022] GET /identity/v3/groups/02deab00a40e4493bb6ddac19fc25741 => generated 226 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:11.826746 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93400d27-0d00-4a66-90cd-ba4a4ab5f251 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:11.827266 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93400d27-0d00-4a66-90cd-ba4a4ab5f251 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:11.827830 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93400d27-0d00-4a66-90cd-ba4a4ab5f251 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:11.837731 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:11.848302 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-93400d27-0d00-4a66-90cd-ba4a4ab5f251 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:15:11.850525 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-93400d27-0d00-4a66-90cd-ba4a4ab5f251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:15:11.903327 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9cbb01-c1c5-435e-a491-fd88380a19f9 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:11.940511 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 207/431] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:15:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 02:15:11.941747 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93400d27-0d00-4a66-90cd-ba4a4ab5f251 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:11.947558 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c672fd31-a17f-4be0-b6ec-a6d9ded0b381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:11.948230 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c672fd31-a17f-4be0-b6ec-a6d9ded0b381 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:11.948511 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c672fd31-a17f-4be0-b6ec-a6d9ded0b381 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:11.948786 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c672fd31-a17f-4be0-b6ec-a6d9ded0b381 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:11.949621 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c672fd31-a17f-4be0-b6ec-a6d9ded0b381 None admin] RBAC: Adding query filter params (group.id=02deab00a40e4493bb6ddac19fc25741, role.id=65feabf3b27944a8b01a86c8ac608457, scope.project.id=a5370165a2354ba6aec1bb43138f4838) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:11.949928 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c672fd31-a17f-4be0-b6ec-a6d9ded0b381 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:11.954217 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c672fd31-a17f-4be0-b6ec-a6d9ded0b381 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:11.974114 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 225/432] 10.222.0.66 () {56 vars in 1376 bytes} [Mon Jul 18 02:15:11 2022] GET /identity/v3/role_assignments?group.id=02deab00a40e4493bb6ddac19fc25741&scope.project.id=a5370165a2354ba6aec1bb43138f4838&role.id=65feabf3b27944a8b01a86c8ac608457 => generated 611 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:11.976146 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c672fd31-a17f-4be0-b6ec-a6d9ded0b381 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:12.149511 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93400d27-0d00-4a66-90cd-ba4a4ab5f251 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:12.164881 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:12.179280 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c672fd31-a17f-4be0-b6ec-a6d9ded0b381 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:12.199639 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:12.199845 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00f20cf-dd97-438d-b534-dab1f104a6e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:12.199845 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00f20cf-dd97-438d-b534-dab1f104a6e3 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:12.199845 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00f20cf-dd97-438d-b534-dab1f104a6e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:12.199845 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00f20cf-dd97-438d-b534-dab1f104a6e3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:12.200148 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00f20cf-dd97-438d-b534-dab1f104a6e3 None admin] RBAC: Adding query filter params (group.id=02deab00a40e4493bb6ddac19fc25741, role.id=a5b895323e7b4f0db2491abe9ad57ee9, scope.project.id=a5370165a2354ba6aec1bb43138f4838) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:12.200148 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00f20cf-dd97-438d-b534-dab1f104a6e3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:12.200148 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00f20cf-dd97-438d-b534-dab1f104a6e3 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:12.200148 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 208/433] 10.222.0.66 () {56 vars in 1376 bytes} [Mon Jul 18 02:15:11 2022] GET /identity/v3/role_assignments?group.id=02deab00a40e4493bb6ddac19fc25741&scope.project.id=a5370165a2354ba6aec1bb43138f4838&role.id=a5b895323e7b4f0db2491abe9ad57ee9 => generated 611 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:12.221819 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f5ccda-48c9-45dc-842e-1bce8e7b5ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:12.223028 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f5ccda-48c9-45dc-842e-1bce8e7b5ae6 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:12.223521 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f5ccda-48c9-45dc-842e-1bce8e7b5ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:12.223992 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f5ccda-48c9-45dc-842e-1bce8e7b5ae6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:12.237104 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f5ccda-48c9-45dc-842e-1bce8e7b5ae6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 02:15:12.245740 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-c9f5ccda-48c9-45dc-842e-1bce8e7b5ae6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 02:15:12.245740 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:12.245740 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:12.245740 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:12.245740 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:12.245740 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:12.245740 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:12.245740 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:12.245740 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:12.245740 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:12.245740 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:12.245740 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:12.245740 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 02:15:12.245740 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 02:15:12.245740 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 02:15:12.245740 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 02:15:12.246785 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 02:15:12.246785 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 02:15:12.246785 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 02:15:12.246785 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 02:15:12.246785 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:12.246785 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:12.246785 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:12.246785 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:12.246785 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:12.246785 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:12.246785 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:12.246785 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:12.246785 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:12.246785 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:12.246785 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:12.246785 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:12.247586 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:12.247586 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:12.247586 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:12.247586 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:12.247586 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 02:15:12.247586 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 02:15:12.247586 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 02:15:12.247586 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 02:15:12.247586 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 02:15:12.247586 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 02:15:12.247586 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 02:15:12.247586 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:15:12.247586 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 226/434] 10.222.0.66 () {56 vars in 1096 bytes} [Mon Jul 18 02:15:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 02:15:12.255456 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00f20cf-dd97-438d-b534-dab1f104a6e3 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:12.340342 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4894fc-4a38-48d8-8b47-e7b34c7c19ad None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:12.340648 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4894fc-4a38-48d8-8b47-e7b34c7c19ad None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:12.340928 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4894fc-4a38-48d8-8b47-e7b34c7c19ad None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:12.342612 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 227/435] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:15:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 02:15:12.354006 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527837bb-36e0-483b-adfd-c7c35884ff04 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:12.354351 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527837bb-36e0-483b-adfd-c7c35884ff04 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:12.354607 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527837bb-36e0-483b-adfd-c7c35884ff04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:12.376790 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-527837bb-36e0-483b-adfd-c7c35884ff04 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:15:12.379420 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-527837bb-36e0-483b-adfd-c7c35884ff04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:15:12.381601 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00f20cf-dd97-438d-b534-dab1f104a6e3 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:12.429884 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b5e930-d0fd-416a-9e47-45006ca727b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:12.431571 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b5e930-d0fd-416a-9e47-45006ca727b0 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:12.432069 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b5e930-d0fd-416a-9e47-45006ca727b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:12.432538 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b5e930-d0fd-416a-9e47-45006ca727b0 None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:12.433488 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b5e930-d0fd-416a-9e47-45006ca727b0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:12.433990 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b5e930-d0fd-416a-9e47-45006ca727b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:12.436261 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b5e930-d0fd-416a-9e47-45006ca727b0 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:12.450309 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 209/436] 10.222.0.66 () {56 vars in 1099 bytes} [Mon Jul 18 02:15:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:12.456970 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b5e930-d0fd-416a-9e47-45006ca727b0 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:12.472439 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 228/437] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:15:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 02:15:12.509024 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:12.509373 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527837bb-36e0-483b-adfd-c7c35884ff04 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:12.591237 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b5e930-d0fd-416a-9e47-45006ca727b0 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:12.638452 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:12.641252 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899eeb13-afb5-48c0-b860-594314b24cd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:12.642186 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899eeb13-afb5-48c0-b860-594314b24cd0 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:12.642686 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899eeb13-afb5-48c0-b860-594314b24cd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:12.643132 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899eeb13-afb5-48c0-b860-594314b24cd0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:12.650990 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527837bb-36e0-483b-adfd-c7c35884ff04 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:12.688097 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29349035-c8e6-4c64-876d-2e469b428fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:12.689158 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29349035-c8e6-4c64-876d-2e469b428fb4 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:12.689563 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29349035-c8e6-4c64-876d-2e469b428fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:12.689984 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29349035-c8e6-4c64-876d-2e469b428fb4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:12.700674 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29349035-c8e6-4c64-876d-2e469b428fb4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 02:15:12.709729 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-29349035-c8e6-4c64-876d-2e469b428fb4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 02:15:12.709729 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:12.709729 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:12.709729 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:12.709729 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:12.709729 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:12.709729 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:12.709729 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:12.709729 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:12.709729 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:12.709729 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:12.709729 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:12.709729 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 02:15:12.709729 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 02:15:12.709729 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 02:15:12.709729 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 02:15:12.710873 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 02:15:12.710873 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 02:15:12.710873 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 02:15:12.710873 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 02:15:12.710873 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:12.710873 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:12.710873 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:12.710873 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:12.710873 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:12.710873 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:12.710873 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:12.710873 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:12.710873 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:12.710873 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:12.710873 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:12.710873 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:12.711868 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:12.711868 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:12.711868 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:12.711868 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:12.711868 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 02:15:12.711868 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 02:15:12.711868 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 02:15:12.711868 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 02:15:12.711868 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 02:15:12.711868 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 02:15:12.711868 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 02:15:12.711868 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:15:12.713954 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 229/438] 10.222.0.66 () {56 vars in 1099 bytes} [Mon Jul 18 02:15:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 02:15:12.722565 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29349035-c8e6-4c64-876d-2e469b428fb4 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:12.738629 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899eeb13-afb5-48c0-b860-594314b24cd0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:12.740277 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899eeb13-afb5-48c0-b860-594314b24cd0 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:12.792978 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:12.831950 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29349035-c8e6-4c64-876d-2e469b428fb4 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:12.868194 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 210/439] 10.222.0.66 () {58 vars in 1282 bytes} [Mon Jul 18 02:15:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 18 02:15:12.873025 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758fd49f-87c2-45f8-be7d-7ab3e6963b41 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:12.873025 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758fd49f-87c2-45f8-be7d-7ab3e6963b41 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:12.873025 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758fd49f-87c2-45f8-be7d-7ab3e6963b41 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:12.873025 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 211/440] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:15:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:15:12.875091 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758fd49f-87c2-45f8-be7d-7ab3e6963b41 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:12.876798 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ae1d79-ee2a-4aa8-8705-f129e10f2412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:12.877484 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ae1d79-ee2a-4aa8-8705-f129e10f2412 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:12.877899 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ae1d79-ee2a-4aa8-8705-f129e10f2412 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:12.878222 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ae1d79-ee2a-4aa8-8705-f129e10f2412 None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:12.879122 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ae1d79-ee2a-4aa8-8705-f129e10f2412 None admin] RBAC: Adding query filter params (name=member) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:12.879434 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ae1d79-ee2a-4aa8-8705-f129e10f2412 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:12.881096 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ae1d79-ee2a-4aa8-8705-f129e10f2412 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:12.891135 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 230/441] 10.222.0.66 () {56 vars in 1101 bytes} [Mon Jul 18 02:15:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 02:15:12.892464 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bdc108-85e8-442f-ad17-fe70f7c7537e None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:12.892756 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bdc108-85e8-442f-ad17-fe70f7c7537e None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:12.893027 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bdc108-85e8-442f-ad17-fe70f7c7537e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:12.910270 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-f4bdc108-85e8-442f-ad17-fe70f7c7537e None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:15:12.911472 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-f4bdc108-85e8-442f-ad17-fe70f7c7537e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:15:12.924657 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:12.977125 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 231/442] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:15:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 02:15:12.977908 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bdc108-85e8-442f-ad17-fe70f7c7537e None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:12.981072 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758fd49f-87c2-45f8-be7d-7ab3e6963b41 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:13.015198 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17cdbee-8b7d-4a8b-b070-45f8a150be76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:13.016051 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17cdbee-8b7d-4a8b-b070-45f8a150be76 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:13.016331 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17cdbee-8b7d-4a8b-b070-45f8a150be76 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:13.016610 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17cdbee-8b7d-4a8b-b070-45f8a150be76 None admin] PATH_INFO: `/v3/projects/a05e40a15656441b94f5763e4d68d305` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:13.017776 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17cdbee-8b7d-4a8b-b070-45f8a150be76 None admin] RBAC: Authorizing `identity:get_project(project_id=a05e40a15656441b94f5763e4d68d305)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:13.019256 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17cdbee-8b7d-4a8b-b070-45f8a150be76 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:13.020946 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 212/443] 10.222.0.66 () {56 vars in 1186 bytes} [Mon Jul 18 02:15:12 2022] GET /identity/v3/projects/a05e40a15656441b94f5763e4d68d305 => generated 344 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:13.022937 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17cdbee-8b7d-4a8b-b070-45f8a150be76 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:13.034056 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:13.084906 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bdc108-85e8-442f-ad17-fe70f7c7537e None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:13.117933 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab58bad-3ee8-4ef2-9e21-302de4bb3953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:13.118599 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab58bad-3ee8-4ef2-9e21-302de4bb3953 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:13.118884 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab58bad-3ee8-4ef2-9e21-302de4bb3953 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:13.119160 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab58bad-3ee8-4ef2-9e21-302de4bb3953 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:13.121645 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17cdbee-8b7d-4a8b-b070-45f8a150be76 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:13.136767 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:13.156431 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cda120-9688-42e2-818e-b9851b801092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:13.157230 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cda120-9688-42e2-818e-b9851b801092 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:13.157522 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cda120-9688-42e2-818e-b9851b801092 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:13.157800 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cda120-9688-42e2-818e-b9851b801092 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:13.167943 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cda120-9688-42e2-818e-b9851b801092 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 02:15:13.176298 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-71cda120-9688-42e2-818e-b9851b801092 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 02:15:13.176298 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:13.176298 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:13.176298 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:13.176298 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:13.176298 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:13.176298 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:13.176298 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:13.176298 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:13.176298 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:13.176298 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:13.176298 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:13.176298 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 02:15:13.176298 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 02:15:13.176298 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 02:15:13.176298 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 02:15:13.177208 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 02:15:13.177208 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 02:15:13.177208 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 02:15:13.177208 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 02:15:13.177208 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:13.177208 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:13.177208 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:13.177208 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:13.177208 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:13.177208 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:13.177208 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:13.177208 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:13.177208 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:13.177208 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:13.177208 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:13.177208 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:13.178026 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:13.178026 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:13.178026 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:13.178026 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:13.178026 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 02:15:13.178026 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 02:15:13.178026 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 02:15:13.178026 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 02:15:13.178026 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 02:15:13.178026 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 02:15:13.178026 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 02:15:13.178026 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:15:13.178026 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 213/444] 10.222.0.66 () {56 vars in 1114 bytes} [Mon Jul 18 02:15:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:13.178744 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cda120-9688-42e2-818e-b9851b801092 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:13.192236 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab58bad-3ee8-4ef2-9e21-302de4bb3953 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:13.193700 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab58bad-3ee8-4ef2-9e21-302de4bb3953 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:13.243686 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:13.276789 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cda120-9688-42e2-818e-b9851b801092 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:13.290403 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 232/445] 10.222.0.66 () {58 vars in 1282 bytes} [Mon Jul 18 02:15:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 02:15:13.291227 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab58bad-3ee8-4ef2-9e21-302de4bb3953 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:13.317453 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c68684e-dbd9-4d3a-9fa2-0a386e637dda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:13.318378 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c68684e-dbd9-4d3a-9fa2-0a386e637dda None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:13.318671 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c68684e-dbd9-4d3a-9fa2-0a386e637dda None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:13.318947 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c68684e-dbd9-4d3a-9fa2-0a386e637dda None admin] PATH_INFO: `/v3/groups/e429b16d0a744db79625f143ae9e4696` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:13.320115 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c68684e-dbd9-4d3a-9fa2-0a386e637dda None admin] RBAC: Authorizing `identity:get_group(group_id=e429b16d0a744db79625f143ae9e4696)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:13.321572 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c68684e-dbd9-4d3a-9fa2-0a386e637dda None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:13.323315 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 214/446] 10.222.0.66 () {56 vars in 1180 bytes} [Mon Jul 18 02:15:13 2022] GET /identity/v3/groups/e429b16d0a744db79625f143ae9e4696 => generated 229 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:13.324209 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c68684e-dbd9-4d3a-9fa2-0a386e637dda None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:13.339590 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:13.394143 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab58bad-3ee8-4ef2-9e21-302de4bb3953 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:13.427014 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c68684e-dbd9-4d3a-9fa2-0a386e637dda None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:13.428498 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fb3a7b-c069-4d32-9da4-1a449201ea88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:13.429157 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fb3a7b-c069-4d32-9da4-1a449201ea88 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:13.429439 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fb3a7b-c069-4d32-9da4-1a449201ea88 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:13.429710 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fb3a7b-c069-4d32-9da4-1a449201ea88 None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:13.430481 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fb3a7b-c069-4d32-9da4-1a449201ea88 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:13.430786 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fb3a7b-c069-4d32-9da4-1a449201ea88 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:13.432150 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fb3a7b-c069-4d32-9da4-1a449201ea88 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:13.440028 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 233/447] 10.222.0.66 () {56 vars in 1111 bytes} [Mon Jul 18 02:15:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:13.440810 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fb3a7b-c069-4d32-9da4-1a449201ea88 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:13.460966 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027a8c08-e641-4ae6-86de-4b0e133b5a6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:13.461825 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027a8c08-e641-4ae6-86de-4b0e133b5a6d None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:13.462133 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027a8c08-e641-4ae6-86de-4b0e133b5a6d None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:13.462413 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027a8c08-e641-4ae6-86de-4b0e133b5a6d None admin] PATH_INFO: `/v3/projects/9748772803a44ccea6827637294a3a97` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:13.463627 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027a8c08-e641-4ae6-86de-4b0e133b5a6d None admin] RBAC: Authorizing `identity:get_project(project_id=9748772803a44ccea6827637294a3a97)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:13.465088 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027a8c08-e641-4ae6-86de-4b0e133b5a6d None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:13.466815 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 215/448] 10.222.0.66 () {56 vars in 1186 bytes} [Mon Jul 18 02:15:13 2022] GET /identity/v3/projects/9748772803a44ccea6827637294a3a97 => generated 302 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:13.467569 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027a8c08-e641-4ae6-86de-4b0e133b5a6d None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:13.472003 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:13.543288 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fb3a7b-c069-4d32-9da4-1a449201ea88 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:13.554128 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:13.573582 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027a8c08-e641-4ae6-86de-4b0e133b5a6d None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:13.576680 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7cdf22-7cf1-4d57-9326-096d76cf0d09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:13.577986 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7cdf22-7cf1-4d57-9326-096d76cf0d09 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:13.578299 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7cdf22-7cf1-4d57-9326-096d76cf0d09 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:13.578578 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7cdf22-7cf1-4d57-9326-096d76cf0d09 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:13.579409 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7cdf22-7cf1-4d57-9326-096d76cf0d09 None admin] RBAC: Adding query filter params (group.id=e429b16d0a744db79625f143ae9e4696, role.id=d4aa3324f0c84887b41d0dc4acf7f30e, scope.project.id=a05e40a15656441b94f5763e4d68d305) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:13.579725 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7cdf22-7cf1-4d57-9326-096d76cf0d09 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:13.581213 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7cdf22-7cf1-4d57-9326-096d76cf0d09 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:13.587101 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 234/449] 10.222.0.66 () {56 vars in 1376 bytes} [Mon Jul 18 02:15:13 2022] GET /identity/v3/role_assignments?group.id=e429b16d0a744db79625f143ae9e4696&scope.project.id=a05e40a15656441b94f5763e4d68d305&role.id=d4aa3324f0c84887b41d0dc4acf7f30e => generated 611 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:13.587879 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7cdf22-7cf1-4d57-9326-096d76cf0d09 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:13.625618 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd453a1e-5f3d-4ed6-bea4-c0c597e32b5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:13.626453 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd453a1e-5f3d-4ed6-bea4-c0c597e32b5a None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:13.626799 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd453a1e-5f3d-4ed6-bea4-c0c597e32b5a None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:13.627080 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd453a1e-5f3d-4ed6-bea4-c0c597e32b5a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:13.708577 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd453a1e-5f3d-4ed6-bea4-c0c597e32b5a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:13.710383 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd453a1e-5f3d-4ed6-bea4-c0c597e32b5a None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:13.713841 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:13.718563 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7cdf22-7cf1-4d57-9326-096d76cf0d09 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:13.760934 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28eead6e-a053-4764-98ed-1f3a0e665d14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:13.761793 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eead6e-a053-4764-98ed-1f3a0e665d14 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:13.762189 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eead6e-a053-4764-98ed-1f3a0e665d14 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:13.763153 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eead6e-a053-4764-98ed-1f3a0e665d14 None admin] PATH_INFO: `/v3/groups/02deab00a40e4493bb6ddac19fc25741` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:13.766119 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28eead6e-a053-4764-98ed-1f3a0e665d14 None admin] RBAC: Authorizing `identity:get_group(group_id=02deab00a40e4493bb6ddac19fc25741)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:13.767575 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28eead6e-a053-4764-98ed-1f3a0e665d14 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:13.769201 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 235/450] 10.222.0.66 () {56 vars in 1180 bytes} [Mon Jul 18 02:15:13 2022] GET /identity/v3/groups/02deab00a40e4493bb6ddac19fc25741 => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:13.775077 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28eead6e-a053-4764-98ed-1f3a0e665d14 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:13.821706 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:13.823776 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 216/451] 10.222.0.66 () {58 vars in 1282 bytes} [Mon Jul 18 02:15:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 02:15:13.830131 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd453a1e-5f3d-4ed6-bea4-c0c597e32b5a None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:13.877559 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28eead6e-a053-4764-98ed-1f3a0e665d14 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:13.911350 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a502c2df-58e4-4c2a-87bd-d9f6816ddc87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:13.911992 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a502c2df-58e4-4c2a-87bd-d9f6816ddc87 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:13.912270 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a502c2df-58e4-4c2a-87bd-d9f6816ddc87 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:13.912540 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a502c2df-58e4-4c2a-87bd-d9f6816ddc87 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:13.913349 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a502c2df-58e4-4c2a-87bd-d9f6816ddc87 None admin] RBAC: Adding query filter params (group.id=02deab00a40e4493bb6ddac19fc25741, role.id=65feabf3b27944a8b01a86c8ac608457, scope.project.id=9748772803a44ccea6827637294a3a97) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:13.913677 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a502c2df-58e4-4c2a-87bd-d9f6816ddc87 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:13.915134 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a502c2df-58e4-4c2a-87bd-d9f6816ddc87 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:13.920772 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 236/452] 10.222.0.66 () {56 vars in 1376 bytes} [Mon Jul 18 02:15:13 2022] GET /identity/v3/role_assignments?group.id=02deab00a40e4493bb6ddac19fc25741&scope.project.id=9748772803a44ccea6827637294a3a97&role.id=65feabf3b27944a8b01a86c8ac608457 => generated 611 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 02:15:13.929835 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd453a1e-5f3d-4ed6-bea4-c0c597e32b5a None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:13.932455 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:13.968761 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b689348-9c85-47f4-94af-d4cec6aa0eae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:13.969644 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b689348-9c85-47f4-94af-d4cec6aa0eae None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:13.969923 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b689348-9c85-47f4-94af-d4cec6aa0eae None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:13.970233 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b689348-9c85-47f4-94af-d4cec6aa0eae None admin] PATH_INFO: `/v3/projects/9748772803a44ccea6827637294a3a97` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:13.971546 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b689348-9c85-47f4-94af-d4cec6aa0eae None admin] RBAC: Authorizing `identity:get_project(project_id=9748772803a44ccea6827637294a3a97)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:13.973037 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b689348-9c85-47f4-94af-d4cec6aa0eae None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:13.974884 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 217/453] 10.222.0.66 () {56 vars in 1186 bytes} [Mon Jul 18 02:15:13 2022] GET /identity/v3/projects/9748772803a44ccea6827637294a3a97 => generated 302 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 02:15:13.980662 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a502c2df-58e4-4c2a-87bd-d9f6816ddc87 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:14.085684 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a502c2df-58e4-4c2a-87bd-d9f6816ddc87 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:14.092655 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:14.120124 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0808cce-2b80-4a70-91bd-ef9a67f05b30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:14.120981 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0808cce-2b80-4a70-91bd-ef9a67f05b30 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:14.121262 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0808cce-2b80-4a70-91bd-ef9a67f05b30 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:14.121611 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0808cce-2b80-4a70-91bd-ef9a67f05b30 None admin] PATH_INFO: `/v3/groups/02deab00a40e4493bb6ddac19fc25741` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:14.122825 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0808cce-2b80-4a70-91bd-ef9a67f05b30 None admin] RBAC: Authorizing `identity:get_group(group_id=02deab00a40e4493bb6ddac19fc25741)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:14.124374 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0808cce-2b80-4a70-91bd-ef9a67f05b30 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:14.126736 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 237/454] 10.222.0.66 () {56 vars in 1180 bytes} [Mon Jul 18 02:15:13 2022] GET /identity/v3/groups/02deab00a40e4493bb6ddac19fc25741 => generated 226 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:14.132733 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b689348-9c85-47f4-94af-d4cec6aa0eae None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:14.234981 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b689348-9c85-47f4-94af-d4cec6aa0eae None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:14.269114 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b94efa2-bdb5-45c9-9322-7123968e11a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:14.269774 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b94efa2-bdb5-45c9-9322-7123968e11a1 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:14.270059 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b94efa2-bdb5-45c9-9322-7123968e11a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:14.270408 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b94efa2-bdb5-45c9-9322-7123968e11a1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:14.271234 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b94efa2-bdb5-45c9-9322-7123968e11a1 None admin] RBAC: Adding query filter params (group.id=02deab00a40e4493bb6ddac19fc25741, role.id=a5b895323e7b4f0db2491abe9ad57ee9, scope.project.id=9748772803a44ccea6827637294a3a97) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:14.271545 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b94efa2-bdb5-45c9-9322-7123968e11a1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:14.272979 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b94efa2-bdb5-45c9-9322-7123968e11a1 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:14.278977 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 218/455] 10.222.0.66 () {56 vars in 1376 bytes} [Mon Jul 18 02:15:14 2022] GET /identity/v3/role_assignments?group.id=02deab00a40e4493bb6ddac19fc25741&scope.project.id=9748772803a44ccea6827637294a3a97&role.id=a5b895323e7b4f0db2491abe9ad57ee9 => generated 611 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:16.789029 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9391cc7d-48ba-45d6-b5d1-34c77c1f798f None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:16.789551 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9391cc7d-48ba-45d6-b5d1-34c77c1f798f None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:16.789626 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9391cc7d-48ba-45d6-b5d1-34c77c1f798f None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:16.791004 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 238/456] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:15:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:15:16.803904 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa04fe09-a86b-4bf0-ab97-bd45afb5fb73 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:16.804777 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa04fe09-a86b-4bf0-ab97-bd45afb5fb73 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:16.805247 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa04fe09-a86b-4bf0-ab97-bd45afb5fb73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:16.831021 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-fa04fe09-a86b-4bf0-ab97-bd45afb5fb73 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:15:16.832502 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-fa04fe09-a86b-4bf0-ab97-bd45afb5fb73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:15:16.871436 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:16.922211 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 219/457] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:15:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 02:15:16.959395 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71b1017-500e-4be9-9e92-b732b3bf409c None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:16.959701 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71b1017-500e-4be9-9e92-b732b3bf409c None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:16.959951 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71b1017-500e-4be9-9e92-b732b3bf409c None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:16.961560 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 239/458] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:15:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:15:16.966414 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71b1017-500e-4be9-9e92-b732b3bf409c None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:16.967249 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96169488-dcac-44ba-b138-a61fe1874ce8 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:16.968443 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96169488-dcac-44ba-b138-a61fe1874ce8 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:16.969185 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96169488-dcac-44ba-b138-a61fe1874ce8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:16.996908 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-96169488-dcac-44ba-b138-a61fe1874ce8 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:15:16.999547 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-96169488-dcac-44ba-b138-a61fe1874ce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:15:17.126068 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 220/459] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:15:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 02:15:17.154025 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96169488-dcac-44ba-b138-a61fe1874ce8 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:17.167271 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71b1017-500e-4be9-9e92-b732b3bf409c None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:17.225998 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9087a109-ed28-4bcd-b77a-af4d2ea9d449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:17.229948 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9087a109-ed28-4bcd-b77a-af4d2ea9d449 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:17.230273 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9087a109-ed28-4bcd-b77a-af4d2ea9d449 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:17.230553 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9087a109-ed28-4bcd-b77a-af4d2ea9d449 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:17.242309 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9087a109-ed28-4bcd-b77a-af4d2ea9d449 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:17.243770 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9087a109-ed28-4bcd-b77a-af4d2ea9d449 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:17.257658 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:17.261573 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-9087a109-ed28-4bcd-b77a-af4d2ea9d449 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:15:17.261573 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:17.261573 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:17.261573 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:17.261573 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:17.261573 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:17.261573 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:17.261573 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:17.261573 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:17.261573 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:17.261573 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:17.261573 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:17.261573 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 02:15:17.261573 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 02:15:17.261573 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 02:15:17.262450 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 02:15:17.262450 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:17.262450 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:17.262450 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:17.262450 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:17.262450 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:17.262450 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:17.262450 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:17.262450 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:17.262450 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:17.262450 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:17.262450 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:17.262450 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:17.262450 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:17.262450 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:17.262450 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:17.262450 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:17.263324 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 02:15:17.263324 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 02:15:17.263324 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:15:17.263324 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:15:17.263324 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 240/460] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 18 02:15:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:17.266117 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96169488-dcac-44ba-b138-a61fe1874ce8 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:17.271614 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9087a109-ed28-4bcd-b77a-af4d2ea9d449 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:17.301353 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ddf1f8-5638-42bf-bfd9-c5ebad0b7e9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:17.302285 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ddf1f8-5638-42bf-bfd9-c5ebad0b7e9c None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:17.302573 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ddf1f8-5638-42bf-bfd9-c5ebad0b7e9c None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:17.302849 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ddf1f8-5638-42bf-bfd9-c5ebad0b7e9c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:17.307445 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:17.315306 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ddf1f8-5638-42bf-bfd9-c5ebad0b7e9c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:17.316978 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ddf1f8-5638-42bf-bfd9-c5ebad0b7e9c None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:17.335297 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-f2ddf1f8-5638-42bf-bfd9-c5ebad0b7e9c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:15:17.335297 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:17.335297 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:17.335297 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:17.335297 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:17.335297 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:17.335297 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:17.335297 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:17.335297 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:17.335297 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:17.335297 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:17.335297 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:17.335297 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 02:15:17.335297 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 02:15:17.335297 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 02:15:17.337329 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 02:15:17.337329 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:17.337329 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:17.337329 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:17.337329 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:17.337329 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:17.337329 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:17.337329 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:17.337329 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:17.337329 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:17.337329 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:17.337329 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:17.337329 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:17.337329 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:17.337329 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:17.337329 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:17.337329 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:17.339454 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 02:15:17.339454 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 02:15:17.339454 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:15:17.339454 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:15:17.339454 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 221/461] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 18 02:15:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 02:15:17.339454 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d1a689-252d-40b3-9b24-5a59b85390b4 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:17.339454 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d1a689-252d-40b3-9b24-5a59b85390b4 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:17.339454 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d1a689-252d-40b3-9b24-5a59b85390b4 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:17.340495 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 222/462] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:15:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:15:17.344754 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d1a689-252d-40b3-9b24-5a59b85390b4 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:17.402168 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9087a109-ed28-4bcd-b77a-af4d2ea9d449 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:17.445930 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8179cfbe-de76-4ffb-b3fe-23a92a2c2ed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:17.446631 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8179cfbe-de76-4ffb-b3fe-23a92a2c2ed6 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:17.447070 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8179cfbe-de76-4ffb-b3fe-23a92a2c2ed6 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:17.447361 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8179cfbe-de76-4ffb-b3fe-23a92a2c2ed6 None admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:17.448109 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8179cfbe-de76-4ffb-b3fe-23a92a2c2ed6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:17.448748 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8179cfbe-de76-4ffb-b3fe-23a92a2c2ed6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:17.450480 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8179cfbe-de76-4ffb-b3fe-23a92a2c2ed6 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:17.459018 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d1a689-252d-40b3-9b24-5a59b85390b4 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:17.474239 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 241/463] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 18 02:15:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 02:15:17.476231 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a83a39-3b46-41cf-8c50-73abaa8cb728 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:17.476522 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a83a39-3b46-41cf-8c50-73abaa8cb728 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:17.476795 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a83a39-3b46-41cf-8c50-73abaa8cb728 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:17.497065 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-f1a83a39-3b46-41cf-8c50-73abaa8cb728 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:15:17.498290 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-f1a83a39-3b46-41cf-8c50-73abaa8cb728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:15:17.513827 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d0d14b-2f2d-47bc-8262-f9a1d061b179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:17.515452 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d0d14b-2f2d-47bc-8262-f9a1d061b179 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:17.515995 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d0d14b-2f2d-47bc-8262-f9a1d061b179 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:17.516449 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d0d14b-2f2d-47bc-8262-f9a1d061b179 None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:17.517621 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d0d14b-2f2d-47bc-8262-f9a1d061b179 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:17.518498 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d0d14b-2f2d-47bc-8262-f9a1d061b179 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:17.520068 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d0d14b-2f2d-47bc-8262-f9a1d061b179 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:17.538619 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 223/464] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 18 02:15:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:17.539670 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d0d14b-2f2d-47bc-8262-f9a1d061b179 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:17.552861 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:17.582135 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 242/465] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:15:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 02:15:17.582135 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a83a39-3b46-41cf-8c50-73abaa8cb728 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:17.614897 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:17.695302 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d0d14b-2f2d-47bc-8262-f9a1d061b179 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:17.703008 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a83a39-3b46-41cf-8c50-73abaa8cb728 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:17.739323 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab59c35d-c7df-47b4-958e-3e1089e51650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:17.740103 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab59c35d-c7df-47b4-958e-3e1089e51650 None admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:17.740458 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab59c35d-c7df-47b4-958e-3e1089e51650 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:17.740805 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab59c35d-c7df-47b4-958e-3e1089e51650 None admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:17.741257 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf53e44-4bf9-42b0-bc45-723dca199d17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:17.741642 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab59c35d-c7df-47b4-958e-3e1089e51650 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:17.741937 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf53e44-4bf9-42b0-bc45-723dca199d17 None admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:17.742250 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf53e44-4bf9-42b0-bc45-723dca199d17 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:17.742537 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf53e44-4bf9-42b0-bc45-723dca199d17 None admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:17.743248 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf53e44-4bf9-42b0-bc45-723dca199d17 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:17.743397 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab59c35d-c7df-47b4-958e-3e1089e51650 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:17.744682 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf53e44-4bf9-42b0-bc45-723dca199d17 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:17.906133 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 243/466] 10.222.0.66 () {60 vars in 1132 bytes} [Mon Jul 18 02:15:17 2022] POST /identity/v3/users => generated 248 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:15:17.906133 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab59c35d-c7df-47b4-958e-3e1089e51650 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:17.949524 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 224/467] 10.222.0.66 () {60 vars in 1132 bytes} [Mon Jul 18 02:15:17 2022] POST /identity/v3/users => generated 246 bytes in 410 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:15:17.951471 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e7d848-055e-4ef4-b59b-e72356fb08f4 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:17.952288 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e7d848-055e-4ef4-b59b-e72356fb08f4 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:17.952574 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e7d848-055e-4ef4-b59b-e72356fb08f4 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:17.954706 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 225/468] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:15:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:15:17.960307 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5663e169-41d5-446a-b97c-796012ace337 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:17.960622 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5663e169-41d5-446a-b97c-796012ace337 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:17.960921 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5663e169-41d5-446a-b97c-796012ace337 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:17.979735 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-5663e169-41d5-446a-b97c-796012ace337 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:15:17.981330 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-5663e169-41d5-446a-b97c-796012ace337 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:15:17.994434 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:18.032666 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab59c35d-c7df-47b4-958e-3e1089e51650 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:18.052627 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:18.062262 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 226/469] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:15:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 02:15:18.075659 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd274743-b708-4a9a-aa8e-3cbc3c1dc3e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:18.076471 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd274743-b708-4a9a-aa8e-3cbc3c1dc3e6 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:18.080881 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd274743-b708-4a9a-aa8e-3cbc3c1dc3e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:18.080881 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd274743-b708-4a9a-aa8e-3cbc3c1dc3e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:18.089952 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5663e169-41d5-446a-b97c-796012ace337 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:18.101771 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd274743-b708-4a9a-aa8e-3cbc3c1dc3e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:18.103318 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd274743-b708-4a9a-aa8e-3cbc3c1dc3e6 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:18.127148 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-fd274743-b708-4a9a-aa8e-3cbc3c1dc3e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:15:18.127148 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:18.127148 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:18.127148 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:18.127148 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:18.127148 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:18.127148 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:18.127148 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:18.127148 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:18.127148 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:18.127148 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:18.127148 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:18.127148 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 02:15:18.127148 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 02:15:18.127148 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 02:15:18.128092 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 02:15:18.128092 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:18.128092 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:18.128092 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:18.128092 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:18.128092 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:18.128092 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:18.128092 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:18.128092 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:18.128092 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:18.128092 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:18.128092 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:18.128092 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:18.128092 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:18.128092 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:18.128092 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:18.128092 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:18.130511 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 02:15:18.130511 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 02:15:18.130511 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:15:18.130511 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:15:18.130511 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 244/470] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 18 02:15:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:18.130511 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9573b9-6bce-4565-945b-9b228a4baae7 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:18.130511 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9573b9-6bce-4565-945b-9b228a4baae7 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:18.130511 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9573b9-6bce-4565-945b-9b228a4baae7 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:18.135965 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 245/471] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:15:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:15:18.137924 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f596e2af-afa8-46a8-a607-9f6de519e52a None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:18.138246 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f596e2af-afa8-46a8-a607-9f6de519e52a None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:18.138506 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f596e2af-afa8-46a8-a607-9f6de519e52a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:18.158871 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-f596e2af-afa8-46a8-a607-9f6de519e52a None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:15:18.160109 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-f596e2af-afa8-46a8-a607-9f6de519e52a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:15:18.214682 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5663e169-41d5-446a-b97c-796012ace337 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:18.244227 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 246/472] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:15:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 02:15:18.245091 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f596e2af-afa8-46a8-a607-9f6de519e52a None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:18.258463 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3339d1c-a7a8-4a14-95de-49a0d22e9dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:18.259332 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3339d1c-a7a8-4a14-95de-49a0d22e9dcd None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:18.260027 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3339d1c-a7a8-4a14-95de-49a0d22e9dcd None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:18.260361 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3339d1c-a7a8-4a14-95de-49a0d22e9dcd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:18.272812 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3339d1c-a7a8-4a14-95de-49a0d22e9dcd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:18.274948 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3339d1c-a7a8-4a14-95de-49a0d22e9dcd None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:18.294542 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-a3339d1c-a7a8-4a14-95de-49a0d22e9dcd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:15:18.294542 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:18.294542 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:18.294542 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:18.294542 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:18.294542 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:18.294542 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:18.294542 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:18.294542 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:18.294542 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:18.294542 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:18.294542 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:18.294542 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 02:15:18.294542 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 02:15:18.294542 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 02:15:18.295552 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 02:15:18.295552 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:18.295552 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:18.295552 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:18.295552 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:18.295552 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:18.295552 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:18.295552 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:18.295552 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:18.295552 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:18.295552 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:18.295552 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:18.295552 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:18.295552 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:18.295552 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:18.295552 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:18.295552 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:18.296922 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 02:15:18.296922 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 02:15:18.296922 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:15:18.296922 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:15:18.296922 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 227/473] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 18 02:15:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:18.297202 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3339d1c-a7a8-4a14-95de-49a0d22e9dcd None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:18.321558 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:18.356213 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:18.379243 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f596e2af-afa8-46a8-a607-9f6de519e52a None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:18.423542 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3339d1c-a7a8-4a14-95de-49a0d22e9dcd None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:18.425083 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a63d6a6-3211-4bc3-a074-d62f2dc1a647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:18.426308 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a63d6a6-3211-4bc3-a074-d62f2dc1a647 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:18.426601 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a63d6a6-3211-4bc3-a074-d62f2dc1a647 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:18.426878 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a63d6a6-3211-4bc3-a074-d62f2dc1a647 None admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:18.428233 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a63d6a6-3211-4bc3-a074-d62f2dc1a647 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:18.428233 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a63d6a6-3211-4bc3-a074-d62f2dc1a647 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:18.429378 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a63d6a6-3211-4bc3-a074-d62f2dc1a647 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:18.444027 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 247/474] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 18 02:15:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:18.444815 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a63d6a6-3211-4bc3-a074-d62f2dc1a647 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:18.462569 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce03b3e9-2094-4d4c-8648-b836f1bbe80c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:18.463586 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce03b3e9-2094-4d4c-8648-b836f1bbe80c None admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:18.464091 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce03b3e9-2094-4d4c-8648-b836f1bbe80c None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:18.464563 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce03b3e9-2094-4d4c-8648-b836f1bbe80c None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:18.465649 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce03b3e9-2094-4d4c-8648-b836f1bbe80c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:18.467284 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce03b3e9-2094-4d4c-8648-b836f1bbe80c None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:18.518913 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.sql.core [None req-ce03b3e9-2094-4d4c-8648-b836f1bbe80c None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 18 02:15:18.518913 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 18 02:15:18.518913 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [parameters: {'id': '3a875218954148dca4c4267d36479b44', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 18 02:15:18.518913 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80399) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 18 02:15:18.538503 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-ce03b3e9-2094-4d4c-8648-b836f1bbe80c None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 18 02:15:18.538503 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:18.538503 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 18 02:15:18.538503 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 18 02:15:18.538503 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 18 02:15:18.538503 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 18 02:15:18.538503 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 18 02:15:18.538503 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application result = self._query(query) Jul 18 02:15:18.538503 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 18 02:15:18.538503 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application conn.query(q) Jul 18 02:15:18.538503 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 18 02:15:18.538503 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 18 02:15:18.538503 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 18 02:15:18.538503 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application result.read() Jul 18 02:15:18.538503 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 18 02:15:18.539481 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 18 02:15:18.539481 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 18 02:15:18.539481 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 18 02:15:18.539481 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 18 02:15:18.539481 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 18 02:15:18.539481 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 18 02:15:18.539481 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 18 02:15:18.539481 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 18 02:15:18.539481 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:15:18.539481 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 18 02:15:18.539481 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:15:18.539481 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:18.539481 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 18 02:15:18.539481 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 18 02:15:18.539481 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 18 02:15:18.539481 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return ref.to_dict() Jul 18 02:15:18.539481 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 18 02:15:18.539481 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application next(self.gen) Jul 18 02:15:18.540513 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 18 02:15:18.540513 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application yield resource Jul 18 02:15:18.540513 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 18 02:15:18.540513 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application next(self.gen) Jul 18 02:15:18.540513 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 18 02:15:18.540513 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self.session.rollback() Jul 18 02:15:18.540513 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 18 02:15:18.540513 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self.force_reraise() Jul 18 02:15:18.540513 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 18 02:15:18.540513 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise self.value Jul 18 02:15:18.540513 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 18 02:15:18.540513 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 18 02:15:18.540513 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 18 02:15:18.540513 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application session.commit() Jul 18 02:15:18.540513 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 18 02:15:18.540513 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 18 02:15:18.540513 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 18 02:15:18.541404 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._prepare_impl() Jul 18 02:15:18.541404 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 18 02:15:18.541404 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self.session.flush() Jul 18 02:15:18.541404 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 18 02:15:18.541404 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._flush(objects) Jul 18 02:15:18.541404 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 18 02:15:18.541404 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 18 02:15:18.541404 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 18 02:15:18.541404 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application compat.raise_( Jul 18 02:15:18.541404 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 18 02:15:18.541404 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception Jul 18 02:15:18.541404 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 18 02:15:18.541404 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application flush_context.execute() Jul 18 02:15:18.541404 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 18 02:15:18.541404 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rec.execute(self) Jul 18 02:15:18.541404 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 18 02:15:18.541404 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 18 02:15:18.543791 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 18 02:15:18.543791 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 18 02:15:18.543791 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 18 02:15:18.543791 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 18 02:15:18.543791 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 18 02:15:18.543791 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 18 02:15:18.543791 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 18 02:15:18.543791 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 18 02:15:18.543791 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 18 02:15:18.543791 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 18 02:15:18.543791 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 18 02:15:18.543791 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 18 02:15:18.543791 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 18 02:15:18.543791 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 18 02:15:18.543791 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 18 02:15:18.543791 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception Jul 18 02:15:18.545940 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 18 02:15:18.545940 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 18 02:15:18.545940 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 18 02:15:18.545940 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 18 02:15:18.545940 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 18 02:15:18.545940 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application result = self._query(query) Jul 18 02:15:18.545940 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 18 02:15:18.545940 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application conn.query(q) Jul 18 02:15:18.545940 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 18 02:15:18.545940 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 18 02:15:18.545940 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 18 02:15:18.545940 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application result.read() Jul 18 02:15:18.545940 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 18 02:15:18.545940 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 18 02:15:18.545940 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 18 02:15:18.545940 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 18 02:15:18.548675 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 18 02:15:18.548675 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 18 02:15:18.548675 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 18 02:15:18.548675 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 18 02:15:18.548675 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 18 02:15:18.548675 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 18 02:15:18.548675 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [parameters: {'id': '3a875218954148dca4c4267d36479b44', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 18 02:15:18.548675 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 18 02:15:18.548675 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:15:18.548675 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 02:15:18.548675 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:15:18.548675 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:18.548675 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:18.548675 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:18.548675 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:18.548675 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:18.553730 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:18.553730 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:18.553730 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:18.553730 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:18.553730 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:18.553730 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:18.553730 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 18 02:15:18.553730 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 18 02:15:18.553730 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:18.553730 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:18.553730 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 18 02:15:18.553730 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 18 02:15:18.553730 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 18 02:15:18.553730 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 18 02:15:18.553730 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 18 02:15:18.553730 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:15:18.556006 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 228/475] 10.222.0.66 () {60 vars in 1131 bytes} [Mon Jul 18 02:15:18 2022] POST /identity/v3/roles => generated 144 bytes in 243 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:18.556006 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce03b3e9-2094-4d4c-8648-b836f1bbe80c None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:18.578323 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a63d6a6-3211-4bc3-a074-d62f2dc1a647 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:18.597681 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:18.629601 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bec7335-6e81-42cb-83ae-ceae3d3f15a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:18.631050 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bec7335-6e81-42cb-83ae-ceae3d3f15a9 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:18.631852 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bec7335-6e81-42cb-83ae-ceae3d3f15a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:18.632380 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bec7335-6e81-42cb-83ae-ceae3d3f15a9 None admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:18.633967 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bec7335-6e81-42cb-83ae-ceae3d3f15a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:18.634779 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bec7335-6e81-42cb-83ae-ceae3d3f15a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:18.636437 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bec7335-6e81-42cb-83ae-ceae3d3f15a9 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:18.652338 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 248/476] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 18 02:15:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:18.653258 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bec7335-6e81-42cb-83ae-ceae3d3f15a9 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:18.687320 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce03b3e9-2094-4d4c-8648-b836f1bbe80c None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:18.701471 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:18.739946 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3292342-6365-4828-9169-089dd4786ad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:18.741411 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3292342-6365-4828-9169-089dd4786ad6 None admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:18.741972 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3292342-6365-4828-9169-089dd4786ad6 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:18.742473 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3292342-6365-4828-9169-089dd4786ad6 None admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:18.744156 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3292342-6365-4828-9169-089dd4786ad6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:18.746072 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3292342-6365-4828-9169-089dd4786ad6 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:18.767163 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bec7335-6e81-42cb-83ae-ceae3d3f15a9 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:18.804516 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c8646f-b54a-4b0a-bae8-27879bc9111a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:18.805327 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c8646f-b54a-4b0a-bae8-27879bc9111a None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:18.805613 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c8646f-b54a-4b0a-bae8-27879bc9111a None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:18.805890 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c8646f-b54a-4b0a-bae8-27879bc9111a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:18.817380 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c8646f-b54a-4b0a-bae8-27879bc9111a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 02:15:18.825568 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-78c8646f-b54a-4b0a-bae8-27879bc9111a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 02:15:18.825568 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:18.825568 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:18.825568 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:18.825568 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:18.825568 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:18.825568 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:18.825568 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:18.825568 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:18.825568 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:18.825568 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:18.825568 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:18.825568 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 02:15:18.825568 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 02:15:18.825568 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 02:15:18.825568 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 02:15:18.827362 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 02:15:18.827362 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 02:15:18.827362 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 02:15:18.827362 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 02:15:18.827362 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:18.827362 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:18.827362 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:18.827362 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:18.827362 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:18.827362 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:18.827362 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:18.827362 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:18.827362 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:18.827362 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:18.827362 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:18.827362 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:18.828217 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:18.828217 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:18.828217 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:18.828217 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:18.828217 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 02:15:18.828217 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 02:15:18.828217 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 02:15:18.828217 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 02:15:18.828217 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 02:15:18.828217 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 02:15:18.828217 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 02:15:18.828217 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:15:18.828217 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 249/477] 10.222.0.66 () {56 vars in 1114 bytes} [Mon Jul 18 02:15:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:18.828217 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c8646f-b54a-4b0a-bae8-27879bc9111a None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:18.865305 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 229/478] 10.222.0.66 () {60 vars in 1132 bytes} [Mon Jul 18 02:15:18 2022] POST /identity/v3/users => generated 248 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:15:18.866250 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:18.867569 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3292342-6365-4828-9169-089dd4786ad6 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:19.000609 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3292342-6365-4828-9169-089dd4786ad6 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:19.042516 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c8646f-b54a-4b0a-bae8-27879bc9111a None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:19.046000 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d34d1a-9ee9-4ed5-ab60-b621f7757977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:19.046964 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d34d1a-9ee9-4ed5-ab60-b621f7757977 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:19.047513 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d34d1a-9ee9-4ed5-ab60-b621f7757977 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:19.047994 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d34d1a-9ee9-4ed5-ab60-b621f7757977 None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:19.049249 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d34d1a-9ee9-4ed5-ab60-b621f7757977 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:19.050330 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d34d1a-9ee9-4ed5-ab60-b621f7757977 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:19.051960 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d34d1a-9ee9-4ed5-ab60-b621f7757977 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:19.064748 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 230/479] 10.222.0.66 () {56 vars in 1111 bytes} [Mon Jul 18 02:15:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:19.104731 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70129a85-8ddd-4bd5-a662-85f0fec9f917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:19.105410 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70129a85-8ddd-4bd5-a662-85f0fec9f917 None admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:19.105693 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70129a85-8ddd-4bd5-a662-85f0fec9f917 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:19.105966 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70129a85-8ddd-4bd5-a662-85f0fec9f917 None admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:19.106702 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70129a85-8ddd-4bd5-a662-85f0fec9f917 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:19.108298 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70129a85-8ddd-4bd5-a662-85f0fec9f917 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:19.222473 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 250/480] 10.222.0.66 () {60 vars in 1132 bytes} [Mon Jul 18 02:15:18 2022] POST /identity/v3/users => generated 249 bytes in 395 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:15:20.262832 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4902f003-d62a-43a4-af91-b469d857c7d0 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:20.263356 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4902f003-d62a-43a4-af91-b469d857c7d0 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:20.263356 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4902f003-d62a-43a4-af91-b469d857c7d0 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:20.265832 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 231/481] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:15:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:15:20.271811 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d537bcf1-44f6-4c23-98f8-10934c719173 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:20.272338 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d537bcf1-44f6-4c23-98f8-10934c719173 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:20.272338 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d537bcf1-44f6-4c23-98f8-10934c719173 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:20.290568 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-d537bcf1-44f6-4c23-98f8-10934c719173 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:15:20.291778 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-d537bcf1-44f6-4c23-98f8-10934c719173 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:15:20.333741 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dafefa-e94a-4f0b-aba4-b65a1fdf0df5 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:20.334042 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dafefa-e94a-4f0b-aba4-b65a1fdf0df5 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:20.334338 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dafefa-e94a-4f0b-aba4-b65a1fdf0df5 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:20.335985 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 232/482] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:15:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:15:20.343837 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735919c8-f747-4e9d-be28-22fe165968e9 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:20.344127 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735919c8-f747-4e9d-be28-22fe165968e9 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:20.344411 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735919c8-f747-4e9d-be28-22fe165968e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:20.372072 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-735919c8-f747-4e9d-be28-22fe165968e9 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:15:20.373268 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-735919c8-f747-4e9d-be28-22fe165968e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:15:20.390930 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 251/483] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:15:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 02:15:20.407554 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:20.426366 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d537bcf1-44f6-4c23-98f8-10934c719173 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:20.457527 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:20.497707 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 233/484] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:15:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 02:15:20.549072 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735919c8-f747-4e9d-be28-22fe165968e9 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:20.600189 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d537bcf1-44f6-4c23-98f8-10934c719173 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:20.672112 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44217d56-53e7-4cde-bd90-c7f1a5a4064b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:20.672924 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44217d56-53e7-4cde-bd90-c7f1a5a4064b None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:20.673467 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44217d56-53e7-4cde-bd90-c7f1a5a4064b None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:20.673740 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44217d56-53e7-4cde-bd90-c7f1a5a4064b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:20.692780 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44217d56-53e7-4cde-bd90-c7f1a5a4064b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 02:15:20.700605 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-44217d56-53e7-4cde-bd90-c7f1a5a4064b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 02:15:20.700605 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:20.700605 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:20.700605 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:20.700605 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:20.700605 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:20.700605 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:20.700605 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:20.700605 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:20.700605 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:20.700605 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:20.700605 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:20.700605 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 02:15:20.700605 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 02:15:20.700605 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 02:15:20.700605 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 02:15:20.701614 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 02:15:20.701614 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 02:15:20.701614 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 02:15:20.701614 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 02:15:20.701614 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:20.701614 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:20.701614 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:20.701614 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:20.701614 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:20.701614 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:20.701614 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:20.701614 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:20.701614 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:20.701614 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:20.701614 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:20.701614 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:20.702501 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:20.702501 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:20.702501 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:20.702501 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:20.702501 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 02:15:20.702501 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 02:15:20.702501 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 02:15:20.702501 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 02:15:20.702501 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 02:15:20.702501 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 02:15:20.702501 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 02:15:20.702501 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:15:20.706277 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 252/485] 10.222.0.66 () {56 vars in 1102 bytes} [Mon Jul 18 02:15:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 02:15:20.710945 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44217d56-53e7-4cde-bd90-c7f1a5a4064b None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:20.730131 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735919c8-f747-4e9d-be28-22fe165968e9 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:20.811994 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864c00fd-ae17-4dc9-a8a0-d4c9a7aa1fba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:20.812732 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864c00fd-ae17-4dc9-a8a0-d4c9a7aa1fba None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:20.813019 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864c00fd-ae17-4dc9-a8a0-d4c9a7aa1fba None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:20.813299 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864c00fd-ae17-4dc9-a8a0-d4c9a7aa1fba None admin] PATH_INFO: `/v3/roles/service` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:20.835830 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:20.843620 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864c00fd-ae17-4dc9-a8a0-d4c9a7aa1fba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 02:15:20.856495 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44217d56-53e7-4cde-bd90-c7f1a5a4064b None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:20.859840 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-864c00fd-ae17-4dc9-a8a0-d4c9a7aa1fba None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 02:15:20.859840 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:20.859840 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:20.859840 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:20.859840 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:20.859840 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:20.859840 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:20.859840 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:20.859840 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:20.859840 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:20.859840 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:20.859840 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:20.859840 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 02:15:20.859840 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 02:15:20.859840 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 02:15:20.859840 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 02:15:20.860738 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 02:15:20.860738 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 02:15:20.860738 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 02:15:20.860738 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 02:15:20.860738 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:20.860738 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:20.860738 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:20.860738 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:20.860738 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:20.860738 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:20.860738 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:20.860738 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:20.860738 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:20.860738 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:20.860738 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:20.860738 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:20.861574 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:20.861574 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:20.861574 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:20.861574 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:20.861574 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 02:15:20.861574 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 02:15:20.861574 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 02:15:20.861574 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 02:15:20.861574 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 02:15:20.861574 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 02:15:20.861574 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 02:15:20.861574 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:15:20.861574 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 234/486] 10.222.0.66 () {56 vars in 1102 bytes} [Mon Jul 18 02:15:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 02:15:20.866892 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864c00fd-ae17-4dc9-a8a0-d4c9a7aa1fba None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:20.903562 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:20.918867 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee4ad5b-769d-469d-be8f-130a8cd3a218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:20.919545 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee4ad5b-769d-469d-be8f-130a8cd3a218 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:20.919827 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee4ad5b-769d-469d-be8f-130a8cd3a218 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:20.920103 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee4ad5b-769d-469d-be8f-130a8cd3a218 None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:20.921736 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee4ad5b-769d-469d-be8f-130a8cd3a218 None admin] RBAC: Adding query filter params (name=service) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:20.922051 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee4ad5b-769d-469d-be8f-130a8cd3a218 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:20.923726 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee4ad5b-769d-469d-be8f-130a8cd3a218 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:20.945434 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 253/487] 10.222.0.66 () {56 vars in 1103 bytes} [Mon Jul 18 02:15:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:20.950797 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee4ad5b-769d-469d-be8f-130a8cd3a218 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:21.060414 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864c00fd-ae17-4dc9-a8a0-d4c9a7aa1fba None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:21.097887 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d496c2-89da-4494-bf65-3431ab06fb0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:21.098536 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d496c2-89da-4494-bf65-3431ab06fb0f None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:21.098824 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d496c2-89da-4494-bf65-3431ab06fb0f None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:21.099105 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d496c2-89da-4494-bf65-3431ab06fb0f None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:21.099855 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d496c2-89da-4494-bf65-3431ab06fb0f None admin] RBAC: Adding query filter params (name=service) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:21.100165 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d496c2-89da-4494-bf65-3431ab06fb0f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:21.101685 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d496c2-89da-4494-bf65-3431ab06fb0f None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:21.111772 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 235/488] 10.222.0.66 () {56 vars in 1103 bytes} [Mon Jul 18 02:15:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:21.119278 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d496c2-89da-4494-bf65-3431ab06fb0f None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:21.150676 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee4ad5b-769d-469d-be8f-130a8cd3a218 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:21.209466 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb443a3-524b-4332-88c2-0b0a3ed1c3da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:21.210309 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb443a3-524b-4332-88c2-0b0a3ed1c3da None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:21.210592 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb443a3-524b-4332-88c2-0b0a3ed1c3da None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:21.210866 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb443a3-524b-4332-88c2-0b0a3ed1c3da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:21.231639 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb443a3-524b-4332-88c2-0b0a3ed1c3da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:21.233319 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb443a3-524b-4332-88c2-0b0a3ed1c3da None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:21.237636 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:21.238541 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:21.255476 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-6eb443a3-524b-4332-88c2-0b0a3ed1c3da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:15:21.255476 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:21.255476 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:21.255476 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:21.255476 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:21.255476 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:21.255476 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:21.255476 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:21.255476 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:21.255476 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:21.255476 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:21.255476 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:21.255476 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 02:15:21.255476 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 02:15:21.255476 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 02:15:21.256391 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 02:15:21.256391 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:21.256391 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:21.256391 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:21.256391 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:21.256391 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:21.256391 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:21.256391 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:21.256391 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:21.256391 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:21.256391 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:21.256391 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:21.256391 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:21.256391 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:21.256391 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:21.256391 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:21.256391 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:21.257252 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 02:15:21.257252 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 02:15:21.257252 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:15:21.257252 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:15:21.257252 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 254/489] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 18 02:15:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:21.262454 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb443a3-524b-4332-88c2-0b0a3ed1c3da None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:21.271605 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d496c2-89da-4494-bf65-3431ab06fb0f None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:21.347247 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cb6619-8b30-4046-91eb-d00ab50e242b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:21.348117 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cb6619-8b30-4046-91eb-d00ab50e242b None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:21.348349 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cb6619-8b30-4046-91eb-d00ab50e242b None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:21.348632 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cb6619-8b30-4046-91eb-d00ab50e242b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:21.360163 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cb6619-8b30-4046-91eb-d00ab50e242b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:21.362270 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cb6619-8b30-4046-91eb-d00ab50e242b None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:21.379065 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb443a3-524b-4332-88c2-0b0a3ed1c3da None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:21.382398 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-16cb6619-8b30-4046-91eb-d00ab50e242b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:15:21.382398 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:21.382398 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:21.382398 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:21.382398 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:21.382398 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:21.382398 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:21.382398 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:21.382398 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:21.382398 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:21.382398 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:21.382398 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:21.382398 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 02:15:21.382398 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 02:15:21.382398 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 02:15:21.383274 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 02:15:21.383274 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:21.383274 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:21.383274 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:21.383274 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:21.383274 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:21.383274 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:21.383274 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:21.383274 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:21.383274 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:21.383274 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:21.383274 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:21.383274 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:21.383274 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:21.383274 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:21.383274 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:21.383274 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:21.384244 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 02:15:21.384244 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 02:15:21.384244 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:15:21.384244 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:15:21.384244 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 236/490] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 18 02:15:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:21.385244 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506ac2d9-9c9c-4ce8-98a8-59f987a3d7d1 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:21.385538 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506ac2d9-9c9c-4ce8-98a8-59f987a3d7d1 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:21.385821 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506ac2d9-9c9c-4ce8-98a8-59f987a3d7d1 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:21.392401 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 237/491] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:15:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:15:21.393823 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e86594-3d17-4247-a581-5262982b112d None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:21.394122 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e86594-3d17-4247-a581-5262982b112d None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:21.394508 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e86594-3d17-4247-a581-5262982b112d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:21.412919 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-e2e86594-3d17-4247-a581-5262982b112d None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:15:21.414508 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-e2e86594-3d17-4247-a581-5262982b112d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:15:21.424118 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f7541d-1639-46f2-8edb-f6f880319249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:21.424795 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f7541d-1639-46f2-8edb-f6f880319249 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:21.425083 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f7541d-1639-46f2-8edb-f6f880319249 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:21.425796 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f7541d-1639-46f2-8edb-f6f880319249 None admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:21.426584 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f7541d-1639-46f2-8edb-f6f880319249 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:21.427223 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f7541d-1639-46f2-8edb-f6f880319249 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:21.428905 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f7541d-1639-46f2-8edb-f6f880319249 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:21.443935 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 255/492] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 18 02:15:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 02:15:21.444829 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f7541d-1639-46f2-8edb-f6f880319249 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:21.508643 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:21.512850 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 238/493] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:15:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 02:15:21.514806 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e86594-3d17-4247-a581-5262982b112d None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:21.579997 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f7541d-1639-46f2-8edb-f6f880319249 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:21.585399 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:21.617984 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a4cd13-17d5-47fd-b2a0-d3624dca30ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:21.618874 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a4cd13-17d5-47fd-b2a0-d3624dca30ec None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:21.619213 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a4cd13-17d5-47fd-b2a0-d3624dca30ec None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:21.619558 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a4cd13-17d5-47fd-b2a0-d3624dca30ec None admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:21.620347 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a4cd13-17d5-47fd-b2a0-d3624dca30ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:21.620714 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a4cd13-17d5-47fd-b2a0-d3624dca30ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:21.622292 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a4cd13-17d5-47fd-b2a0-d3624dca30ec None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:21.634945 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e86594-3d17-4247-a581-5262982b112d None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:21.635811 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 256/494] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 18 02:15:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:21.640796 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a4cd13-17d5-47fd-b2a0-d3624dca30ec None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:21.683028 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84308d6e-37cc-47af-9a65-0b108b47edd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:21.684139 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84308d6e-37cc-47af-9a65-0b108b47edd1 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:21.684572 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84308d6e-37cc-47af-9a65-0b108b47edd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:21.685046 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84308d6e-37cc-47af-9a65-0b108b47edd1 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:21.694431 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84308d6e-37cc-47af-9a65-0b108b47edd1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:21.696163 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84308d6e-37cc-47af-9a65-0b108b47edd1 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:21.713235 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-84308d6e-37cc-47af-9a65-0b108b47edd1 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 18 02:15:21.713235 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:21.713235 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:21.713235 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:21.713235 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:21.713235 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:21.713235 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:21.713235 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:21.713235 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:21.713235 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:21.713235 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:21.713235 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:21.713235 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 02:15:21.713235 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 02:15:21.713235 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 02:15:21.713235 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 02:15:21.714417 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:21.714417 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:21.714417 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 02:15:21.714417 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 02:15:21.714417 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 02:15:21.714417 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 02:15:21.714417 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:21.714417 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:21.714417 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:21.714417 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:21.714417 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:21.714417 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:21.714417 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:21.714417 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:21.714417 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:21.714417 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:21.714417 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:21.714417 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:21.715550 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:21.715550 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:21.715550 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 02:15:21.715550 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 02:15:21.715550 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 02:15:21.715550 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 02:15:21.715550 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 02:15:21.715550 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 02:15:21.715550 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 18 02:15:21.715550 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:15:21.722727 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 239/495] 10.222.0.66 () {56 vars in 1099 bytes} [Mon Jul 18 02:15:21 2022] GET /identity/v3/users/glance => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:21.723793 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84308d6e-37cc-47af-9a65-0b108b47edd1 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:21.774453 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a4cd13-17d5-47fd-b2a0-d3624dca30ec None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:21.789277 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:21.810231 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a2ff6b-075c-4f97-ae0d-e13e71f72e9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:21.811356 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a2ff6b-075c-4f97-ae0d-e13e71f72e9d None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:21.811630 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a2ff6b-075c-4f97-ae0d-e13e71f72e9d None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:21.811907 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a2ff6b-075c-4f97-ae0d-e13e71f72e9d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:21.823610 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a2ff6b-075c-4f97-ae0d-e13e71f72e9d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 02:15:21.831997 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-07a2ff6b-075c-4f97-ae0d-e13e71f72e9d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 02:15:21.831997 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:21.831997 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:21.831997 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:21.831997 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:21.831997 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:21.831997 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:21.831997 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:21.831997 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:21.831997 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:21.831997 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:21.831997 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:21.831997 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 02:15:21.831997 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 02:15:21.831997 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 02:15:21.831997 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 02:15:21.833080 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 02:15:21.833080 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 02:15:21.833080 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 02:15:21.833080 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 02:15:21.833080 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:21.833080 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:21.833080 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:21.833080 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:21.833080 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:21.833080 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:21.833080 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:21.833080 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:21.833080 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:21.833080 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:21.833080 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:21.833080 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:21.834122 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:21.834122 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:21.834122 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:21.834122 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:21.834122 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 02:15:21.834122 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 02:15:21.834122 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 02:15:21.834122 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 02:15:21.834122 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 02:15:21.834122 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 02:15:21.834122 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 02:15:21.834122 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:15:21.834791 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 257/496] 10.222.0.66 () {56 vars in 1102 bytes} [Mon Jul 18 02:15:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:21.835376 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a2ff6b-075c-4f97-ae0d-e13e71f72e9d None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:21.837845 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84308d6e-37cc-47af-9a65-0b108b47edd1 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:21.860613 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:21.871964 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11063f6e-3421-4ddf-950d-36fc62fe1f73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:21.872771 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11063f6e-3421-4ddf-950d-36fc62fe1f73 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:21.873121 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11063f6e-3421-4ddf-950d-36fc62fe1f73 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:21.873495 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11063f6e-3421-4ddf-950d-36fc62fe1f73 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:21.881108 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11063f6e-3421-4ddf-950d-36fc62fe1f73 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:21.882718 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11063f6e-3421-4ddf-950d-36fc62fe1f73 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:21.897393 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-11063f6e-3421-4ddf-950d-36fc62fe1f73 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 18 02:15:21.897393 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:21.897393 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:21.897393 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:21.897393 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:21.897393 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:21.897393 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:21.897393 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:21.897393 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:21.897393 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:21.897393 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:21.897393 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:21.897393 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 02:15:21.897393 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 02:15:21.897393 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 02:15:21.897393 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 02:15:21.898365 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:21.898365 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:21.898365 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 02:15:21.898365 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 02:15:21.898365 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 02:15:21.898365 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 02:15:21.898365 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:21.898365 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:21.898365 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:21.898365 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:21.898365 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:21.898365 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:21.898365 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:21.898365 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:21.898365 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:21.898365 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:21.898365 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:21.898365 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:21.899291 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:21.899291 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:21.899291 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 02:15:21.899291 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 02:15:21.899291 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 02:15:21.899291 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 02:15:21.899291 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 02:15:21.899291 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 02:15:21.899291 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 18 02:15:21.899291 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:15:21.899291 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 240/497] 10.222.0.66 () {56 vars in 1093 bytes} [Mon Jul 18 02:15:21 2022] GET /identity/v3/users/nova => generated 82 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:21.899865 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fa925c-038c-46b4-9737-e7e7006c2790 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:21.900084 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fa925c-038c-46b4-9737-e7e7006c2790 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:21.900383 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fa925c-038c-46b4-9737-e7e7006c2790 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:21.901589 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 241/498] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:15:21 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:15:21.902673 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7b253c-e592-4aef-889a-aa9572f59991 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:21.902963 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7b253c-e592-4aef-889a-aa9572f59991 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:21.903507 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7b253c-e592-4aef-889a-aa9572f59991 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:21.906269 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 242/499] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:15:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:15:21.906269 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7b253c-e592-4aef-889a-aa9572f59991 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:21.942036 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a2ff6b-075c-4f97-ae0d-e13e71f72e9d None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:21.975440 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b098cb80-0735-466e-a3cb-7415e5095c24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:21.976109 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b098cb80-0735-466e-a3cb-7415e5095c24 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:21.976390 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b098cb80-0735-466e-a3cb-7415e5095c24 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:21.976667 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b098cb80-0735-466e-a3cb-7415e5095c24 None admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:21.977501 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b098cb80-0735-466e-a3cb-7415e5095c24 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:21.977809 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b098cb80-0735-466e-a3cb-7415e5095c24 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:21.979300 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b098cb80-0735-466e-a3cb-7415e5095c24 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:22.010308 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7b253c-e592-4aef-889a-aa9572f59991 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:22.016463 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 258/500] 10.222.0.66 () {56 vars in 1137 bytes} [Mon Jul 18 02:15:21 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:22.017286 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b098cb80-0735-466e-a3cb-7415e5095c24 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:22.044838 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80f836d-87c2-49a1-adc3-f6b25de83653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:22.046277 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80f836d-87c2-49a1-adc3-f6b25de83653 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:22.046557 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80f836d-87c2-49a1-adc3-f6b25de83653 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:22.046829 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80f836d-87c2-49a1-adc3-f6b25de83653 None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:22.047572 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80f836d-87c2-49a1-adc3-f6b25de83653 None admin] RBAC: Adding query filter params (name=service) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:22.048123 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80f836d-87c2-49a1-adc3-f6b25de83653 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:22.049504 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80f836d-87c2-49a1-adc3-f6b25de83653 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:22.051038 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:22.057294 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 243/501] 10.222.0.66 () {56 vars in 1103 bytes} [Mon Jul 18 02:15:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:22.058653 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1774c3c-b345-4cd9-99b7-9ba5f5e6258e None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:22.058853 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1774c3c-b345-4cd9-99b7-9ba5f5e6258e None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:22.059127 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1774c3c-b345-4cd9-99b7-9ba5f5e6258e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:22.075047 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-f1774c3c-b345-4cd9-99b7-9ba5f5e6258e None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:15:22.076225 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-f1774c3c-b345-4cd9-99b7-9ba5f5e6258e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:15:22.116515 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:22.127206 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b098cb80-0735-466e-a3cb-7415e5095c24 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:22.142094 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 244/502] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:15:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 02:15:22.143376 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f641bd8c-4dfd-4cca-976b-81f3bdb980bf None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:22.143663 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f641bd8c-4dfd-4cca-976b-81f3bdb980bf None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:22.143942 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f641bd8c-4dfd-4cca-976b-81f3bdb980bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:22.157334 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01da64d4-608e-4a6e-a54a-d5859a14530a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:22.158009 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01da64d4-608e-4a6e-a54a-d5859a14530a None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:22.158319 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01da64d4-608e-4a6e-a54a-d5859a14530a None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:22.159192 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01da64d4-608e-4a6e-a54a-d5859a14530a None admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:22.159433 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01da64d4-608e-4a6e-a54a-d5859a14530a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:22.159745 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01da64d4-608e-4a6e-a54a-d5859a14530a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:22.161241 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01da64d4-608e-4a6e-a54a-d5859a14530a None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:22.170122 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-f641bd8c-4dfd-4cca-976b-81f3bdb980bf None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:15:22.171893 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-f641bd8c-4dfd-4cca-976b-81f3bdb980bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:15:22.180348 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 259/503] 10.222.0.66 () {56 vars in 1133 bytes} [Mon Jul 18 02:15:22 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:22.181165 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01da64d4-608e-4a6e-a54a-d5859a14530a None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:22.239041 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 245/504] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:15:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 02:15:22.239875 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f641bd8c-4dfd-4cca-976b-81f3bdb980bf None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:22.272946 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:22.282988 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01da64d4-608e-4a6e-a54a-d5859a14530a None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:22.316709 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0855040-a368-4dcd-9f4b-f60ebdd84edc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:22.317353 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0855040-a368-4dcd-9f4b-f60ebdd84edc None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:22.317638 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0855040-a368-4dcd-9f4b-f60ebdd84edc None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:22.317915 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0855040-a368-4dcd-9f4b-f60ebdd84edc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:22.345514 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f641bd8c-4dfd-4cca-976b-81f3bdb980bf None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:22.348424 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:22.379121 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7c8ee7-dffd-469d-a269-ac15c8568e10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:22.379915 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7c8ee7-dffd-469d-a269-ac15c8568e10 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:22.380203 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7c8ee7-dffd-469d-a269-ac15c8568e10 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:22.380589 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7c8ee7-dffd-469d-a269-ac15c8568e10 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:22.386682 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0855040-a368-4dcd-9f4b-f60ebdd84edc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:22.390604 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0855040-a368-4dcd-9f4b-f60ebdd84edc None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:22.391828 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7c8ee7-dffd-469d-a269-ac15c8568e10 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:22.393292 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7c8ee7-dffd-469d-a269-ac15c8568e10 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:22.410814 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-2a7c8ee7-dffd-469d-a269-ac15c8568e10 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:15:22.410814 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:22.410814 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:22.410814 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:22.410814 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:22.410814 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:22.410814 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:22.410814 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:22.410814 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:22.410814 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:22.410814 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:22.410814 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:22.410814 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 02:15:22.410814 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 02:15:22.410814 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 02:15:22.411682 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 02:15:22.411682 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:22.411682 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:22.411682 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:22.411682 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:22.411682 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:22.411682 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:22.411682 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:22.411682 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:22.411682 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:22.411682 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:22.411682 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:22.411682 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:22.411682 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:22.411682 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:22.411682 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:22.411682 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:22.412534 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 02:15:22.412534 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 02:15:22.412534 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:15:22.412534 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:15:22.412534 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 246/505] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 18 02:15:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:22.412805 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7c8ee7-dffd-469d-a269-ac15c8568e10 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:22.480186 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:22.490966 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 260/506] 10.222.0.66 () {58 vars in 1282 bytes} [Mon Jul 18 02:15:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 02:15:22.491765 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0855040-a368-4dcd-9f4b-f60ebdd84edc None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:22.517202 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7c8ee7-dffd-469d-a269-ac15c8568e10 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:22.550643 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c50761-3db5-4a5b-89cf-a56bd2066e03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:22.551436 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c50761-3db5-4a5b-89cf-a56bd2066e03 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:22.551719 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c50761-3db5-4a5b-89cf-a56bd2066e03 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:22.552002 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c50761-3db5-4a5b-89cf-a56bd2066e03 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:22.562867 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c50761-3db5-4a5b-89cf-a56bd2066e03 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:22.564324 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c50761-3db5-4a5b-89cf-a56bd2066e03 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:22.581675 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-67c50761-3db5-4a5b-89cf-a56bd2066e03 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:15:22.581675 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:22.581675 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:22.581675 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:22.581675 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:22.581675 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:22.581675 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:22.581675 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:22.581675 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:22.581675 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:22.581675 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:22.581675 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:22.581675 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 02:15:22.581675 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 02:15:22.581675 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 02:15:22.582760 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 02:15:22.582760 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:22.582760 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:22.582760 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:22.582760 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:22.582760 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:22.582760 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:22.582760 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:22.582760 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:22.582760 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:22.582760 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:22.582760 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:22.582760 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:22.582760 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:22.582760 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:22.582760 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:22.582760 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:22.583935 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 02:15:22.583935 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 02:15:22.583935 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:15:22.583935 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:15:22.583935 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 247/507] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 18 02:15:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:22.584231 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c50761-3db5-4a5b-89cf-a56bd2066e03 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:22.594698 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0855040-a368-4dcd-9f4b-f60ebdd84edc None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:22.596075 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:22.632942 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3093e6fa-93c3-41b3-8c15-9d0d835ab272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:22.633525 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3093e6fa-93c3-41b3-8c15-9d0d835ab272 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:22.633801 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3093e6fa-93c3-41b3-8c15-9d0d835ab272 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:22.634074 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3093e6fa-93c3-41b3-8c15-9d0d835ab272 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:22.691347 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c50761-3db5-4a5b-89cf-a56bd2066e03 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:22.691556 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:22.701757 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3093e6fa-93c3-41b3-8c15-9d0d835ab272 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:22.705297 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3093e6fa-93c3-41b3-8c15-9d0d835ab272 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:22.725211 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7cf55d-43be-43cd-a1e0-e75c4ff6553a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:22.726006 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7cf55d-43be-43cd-a1e0-e75c4ff6553a None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:22.726316 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7cf55d-43be-43cd-a1e0-e75c4ff6553a None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:22.726600 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7cf55d-43be-43cd-a1e0-e75c4ff6553a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:22.737094 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7cf55d-43be-43cd-a1e0-e75c4ff6553a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 02:15:22.744747 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-dc7cf55d-43be-43cd-a1e0-e75c4ff6553a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 02:15:22.744747 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:22.744747 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:22.744747 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:22.744747 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:22.744747 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:22.744747 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:22.744747 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:22.744747 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:22.744747 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:22.744747 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:22.744747 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:22.744747 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 02:15:22.744747 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 02:15:22.744747 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 02:15:22.744747 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 02:15:22.745880 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 02:15:22.745880 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 02:15:22.745880 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 02:15:22.745880 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 02:15:22.745880 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:22.745880 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:22.745880 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:22.745880 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:22.745880 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:22.745880 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:22.745880 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:22.745880 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:22.745880 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:22.745880 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:22.745880 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:22.745880 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:22.746932 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:22.746932 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:22.746932 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:22.746932 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:22.746932 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 02:15:22.746932 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 02:15:22.746932 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 02:15:22.746932 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 02:15:22.746932 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 02:15:22.746932 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 02:15:22.746932 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 02:15:22.746932 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:15:22.746932 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 248/508] 10.222.0.66 () {56 vars in 1102 bytes} [Mon Jul 18 02:15:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:22.746932 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7cf55d-43be-43cd-a1e0-e75c4ff6553a None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:22.806747 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 261/509] 10.222.0.66 () {58 vars in 1282 bytes} [Mon Jul 18 02:15:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 02:15:22.807592 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3093e6fa-93c3-41b3-8c15-9d0d835ab272 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:22.817923 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:22.851583 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7cf55d-43be-43cd-a1e0-e75c4ff6553a None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:22.884986 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0d9a54-1ec3-4a6b-a09d-80557f22665a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:22.885580 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0d9a54-1ec3-4a6b-a09d-80557f22665a None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:22.885865 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0d9a54-1ec3-4a6b-a09d-80557f22665a None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:22.886160 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0d9a54-1ec3-4a6b-a09d-80557f22665a None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:22.886879 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0d9a54-1ec3-4a6b-a09d-80557f22665a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:22.887186 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0d9a54-1ec3-4a6b-a09d-80557f22665a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:22.888554 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0d9a54-1ec3-4a6b-a09d-80557f22665a None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:22.895553 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:22.900263 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 249/510] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 18 02:15:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:22.901051 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0d9a54-1ec3-4a6b-a09d-80557f22665a None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:22.906764 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3093e6fa-93c3-41b3-8c15-9d0d835ab272 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:22.941654 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c54578-8792-4f80-84bf-55cca3d2aa62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:22.942476 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c54578-8792-4f80-84bf-55cca3d2aa62 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:22.942757 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c54578-8792-4f80-84bf-55cca3d2aa62 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:22.943029 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c54578-8792-4f80-84bf-55cca3d2aa62 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:22.952914 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c54578-8792-4f80-84bf-55cca3d2aa62 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:22.954406 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c54578-8792-4f80-84bf-55cca3d2aa62 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:22.971792 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-60c54578-8792-4f80-84bf-55cca3d2aa62 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:15:22.971792 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:22.971792 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:22.971792 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:22.971792 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:22.971792 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:22.971792 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:22.971792 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:22.971792 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:22.971792 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:22.971792 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:22.971792 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:22.971792 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 02:15:22.971792 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 02:15:22.971792 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 02:15:22.972661 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 02:15:22.972661 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:22.972661 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:22.972661 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:22.972661 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:22.972661 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:22.972661 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:22.972661 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:22.972661 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:22.972661 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:22.972661 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:22.972661 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:22.972661 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:22.972661 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:22.972661 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:22.972661 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:22.972661 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:22.973517 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 02:15:22.973517 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 02:15:22.973517 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:15:22.973517 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:15:22.973517 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 262/511] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 18 02:15:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:22.973776 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c54578-8792-4f80-84bf-55cca3d2aa62 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:22.999811 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0d9a54-1ec3-4a6b-a09d-80557f22665a None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:23.026217 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:23.034239 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c10ae6-5c0f-4704-aa8b-3f7b08a55f26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:23.034902 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c10ae6-5c0f-4704-aa8b-3f7b08a55f26 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:23.035188 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c10ae6-5c0f-4704-aa8b-3f7b08a55f26 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:23.035467 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c10ae6-5c0f-4704-aa8b-3f7b08a55f26 None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:23.036216 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c10ae6-5c0f-4704-aa8b-3f7b08a55f26 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:23.036526 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c10ae6-5c0f-4704-aa8b-3f7b08a55f26 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:23.037931 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c10ae6-5c0f-4704-aa8b-3f7b08a55f26 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:23.049616 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 250/512] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 18 02:15:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:23.050363 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c10ae6-5c0f-4704-aa8b-3f7b08a55f26 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:23.072679 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c54578-8792-4f80-84bf-55cca3d2aa62 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:23.109617 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68759eb1-e3e0-4160-b5b5-144e875406f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:23.110314 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68759eb1-e3e0-4160-b5b5-144e875406f9 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:23.110597 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68759eb1-e3e0-4160-b5b5-144e875406f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:23.110870 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68759eb1-e3e0-4160-b5b5-144e875406f9 None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:23.111604 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68759eb1-e3e0-4160-b5b5-144e875406f9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:23.111909 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68759eb1-e3e0-4160-b5b5-144e875406f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:23.113281 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68759eb1-e3e0-4160-b5b5-144e875406f9 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:23.121255 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 263/513] 10.222.0.66 () {56 vars in 1103 bytes} [Mon Jul 18 02:15:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:23.122029 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68759eb1-e3e0-4160-b5b5-144e875406f9 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:23.126715 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:23.154467 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c10ae6-5c0f-4704-aa8b-3f7b08a55f26 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:23.188458 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465bfafb-aad4-489c-bb89-10358196a60a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:23.189249 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465bfafb-aad4-489c-bb89-10358196a60a None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:23.189535 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465bfafb-aad4-489c-bb89-10358196a60a None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:23.189814 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465bfafb-aad4-489c-bb89-10358196a60a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:23.200997 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465bfafb-aad4-489c-bb89-10358196a60a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:23.203072 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465bfafb-aad4-489c-bb89-10358196a60a None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:23.220767 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-465bfafb-aad4-489c-bb89-10358196a60a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:15:23.220767 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:23.220767 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:23.220767 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:23.220767 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:23.220767 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:23.220767 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:23.220767 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:23.220767 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:23.220767 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:23.220767 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:23.220767 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:23.220767 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 02:15:23.220767 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 02:15:23.220767 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 02:15:23.221790 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 02:15:23.221790 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:23.221790 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:23.221790 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:23.221790 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:23.221790 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:23.221790 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:23.221790 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:23.221790 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:23.221790 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:23.221790 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:23.221790 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:23.221790 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:23.221790 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:23.221790 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:23.221790 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:23.221790 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:23.222692 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 02:15:23.222692 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 02:15:23.222692 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:15:23.222692 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:15:23.222921 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68759eb1-e3e0-4160-b5b5-144e875406f9 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:23.222994 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 251/514] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 18 02:15:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:23.222994 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465bfafb-aad4-489c-bb89-10358196a60a None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:23.257934 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c541321a-b093-4462-9cbf-5e15ac4b35dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:23.258749 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c541321a-b093-4462-9cbf-5e15ac4b35dc None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:23.259033 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c541321a-b093-4462-9cbf-5e15ac4b35dc None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:23.259306 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c541321a-b093-4462-9cbf-5e15ac4b35dc None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:23.264243 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:23.267395 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c541321a-b093-4462-9cbf-5e15ac4b35dc None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:23.268843 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c541321a-b093-4462-9cbf-5e15ac4b35dc None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:23.284357 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-c541321a-b093-4462-9cbf-5e15ac4b35dc None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 18 02:15:23.284357 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:23.284357 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:23.284357 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:23.284357 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:23.284357 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:23.284357 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:23.284357 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:23.284357 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:23.284357 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:23.284357 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:23.284357 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:23.284357 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 02:15:23.284357 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 02:15:23.284357 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 02:15:23.284357 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 02:15:23.285442 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:23.285442 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:23.285442 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 02:15:23.285442 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 02:15:23.285442 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 02:15:23.285442 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 02:15:23.285442 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:23.285442 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:23.285442 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:23.285442 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:23.285442 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:23.285442 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:23.285442 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:23.285442 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:23.285442 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:23.285442 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:23.285442 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:23.285442 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:23.286677 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:23.286677 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:23.286677 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 02:15:23.286677 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 02:15:23.286677 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 02:15:23.286677 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 02:15:23.286677 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 02:15:23.286677 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 02:15:23.286677 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 18 02:15:23.286677 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:15:23.286677 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 264/515] 10.222.0.66 () {56 vars in 1099 bytes} [Mon Jul 18 02:15:23 2022] GET /identity/v3/users/cinder => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:23.286677 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c541321a-b093-4462-9cbf-5e15ac4b35dc None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:23.327022 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465bfafb-aad4-489c-bb89-10358196a60a None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:23.348232 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:23.359894 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654b3052-1698-4ed8-8746-2f27b3802453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:23.360554 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654b3052-1698-4ed8-8746-2f27b3802453 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:23.360910 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654b3052-1698-4ed8-8746-2f27b3802453 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:23.361186 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654b3052-1698-4ed8-8746-2f27b3802453 None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:23.361910 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654b3052-1698-4ed8-8746-2f27b3802453 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:23.362246 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654b3052-1698-4ed8-8746-2f27b3802453 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:23.363650 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654b3052-1698-4ed8-8746-2f27b3802453 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:23.375733 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 252/516] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 18 02:15:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:23.377698 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654b3052-1698-4ed8-8746-2f27b3802453 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:23.385744 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c541321a-b093-4462-9cbf-5e15ac4b35dc None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:23.449522 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a45e41-6e21-487f-95ef-9572f46397a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:23.449522 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a45e41-6e21-487f-95ef-9572f46397a7 None admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:23.449522 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a45e41-6e21-487f-95ef-9572f46397a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:23.449522 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a45e41-6e21-487f-95ef-9572f46397a7 None admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:23.450416 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a45e41-6e21-487f-95ef-9572f46397a7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:23.450416 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a45e41-6e21-487f-95ef-9572f46397a7 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:23.473317 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:23.475993 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654b3052-1698-4ed8-8746-2f27b3802453 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:23.508627 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22e42e9-728f-421c-b35a-1c018046ed3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:23.509407 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22e42e9-728f-421c-b35a-1c018046ed3c None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:23.509690 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22e42e9-728f-421c-b35a-1c018046ed3c None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:23.509966 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22e42e9-728f-421c-b35a-1c018046ed3c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:23.520208 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22e42e9-728f-421c-b35a-1c018046ed3c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:23.523335 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22e42e9-728f-421c-b35a-1c018046ed3c None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:23.538838 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-a22e42e9-728f-421c-b35a-1c018046ed3c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:15:23.538838 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:23.538838 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:23.538838 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:23.538838 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:23.538838 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:23.538838 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:23.538838 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:23.538838 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:23.538838 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:23.538838 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:23.538838 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:23.538838 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 02:15:23.538838 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 02:15:23.538838 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 02:15:23.542992 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 02:15:23.542992 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:23.542992 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:23.542992 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:23.542992 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:23.542992 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:23.542992 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:23.542992 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:23.542992 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:23.542992 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:23.542992 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:23.542992 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:23.542992 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:23.542992 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:23.542992 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:23.542992 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:23.542992 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:23.544077 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 02:15:23.544077 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 02:15:23.544077 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:15:23.544077 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:15:23.544077 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 253/517] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 18 02:15:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:23.544077 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22e42e9-728f-421c-b35a-1c018046ed3c None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:23.562667 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 265/518] 10.222.0.66 () {60 vars in 1132 bytes} [Mon Jul 18 02:15:23 2022] POST /identity/v3/users => generated 247 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:15:23.562667 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a45e41-6e21-487f-95ef-9572f46397a7 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:23.663764 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22e42e9-728f-421c-b35a-1c018046ed3c None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:23.684433 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a45e41-6e21-487f-95ef-9572f46397a7 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:23.704780 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e873f0d-df7b-470a-b68e-623b99b573f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:23.705411 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e873f0d-df7b-470a-b68e-623b99b573f1 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:23.705654 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e873f0d-df7b-470a-b68e-623b99b573f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:23.705931 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e873f0d-df7b-470a-b68e-623b99b573f1 None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:23.706699 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e873f0d-df7b-470a-b68e-623b99b573f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:23.707012 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e873f0d-df7b-470a-b68e-623b99b573f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:23.708409 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e873f0d-df7b-470a-b68e-623b99b573f1 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:23.711427 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:23.729098 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 254/519] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 18 02:15:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:23.729098 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e873f0d-df7b-470a-b68e-623b99b573f1 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:23.731664 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a40266-a31a-4dce-93a6-8dc9b01a3536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:23.733085 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a40266-a31a-4dce-93a6-8dc9b01a3536 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:23.733575 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a40266-a31a-4dce-93a6-8dc9b01a3536 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:23.733849 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a40266-a31a-4dce-93a6-8dc9b01a3536 None admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:23.734738 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a40266-a31a-4dce-93a6-8dc9b01a3536 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:23.735043 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a40266-a31a-4dce-93a6-8dc9b01a3536 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:23.736507 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a40266-a31a-4dce-93a6-8dc9b01a3536 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:23.742246 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:23.757341 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 266/520] 10.222.0.66 () {56 vars in 1137 bytes} [Mon Jul 18 02:15:23 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:23.759133 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a40266-a31a-4dce-93a6-8dc9b01a3536 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:23.840105 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e873f0d-df7b-470a-b68e-623b99b573f1 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:23.865275 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a40266-a31a-4dce-93a6-8dc9b01a3536 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:23.875960 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daef7e34-e150-42a0-a847-61639426f5d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:23.876758 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daef7e34-e150-42a0-a847-61639426f5d4 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:23.877040 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daef7e34-e150-42a0-a847-61639426f5d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:23.877314 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daef7e34-e150-42a0-a847-61639426f5d4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:23.884280 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daef7e34-e150-42a0-a847-61639426f5d4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:23.885756 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daef7e34-e150-42a0-a847-61639426f5d4 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:23.900550 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35baac8f-c17c-4b1b-b0cd-8c832386b1a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:23.900736 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-daef7e34-e150-42a0-a847-61639426f5d4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 02:15:23.900736 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:23.900736 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:23.900736 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:23.900736 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:23.900736 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:23.900736 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:23.900736 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:23.900736 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:23.900736 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:23.900736 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:23.900736 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:23.900736 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 02:15:23.900736 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 02:15:23.900736 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 02:15:23.901565 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 02:15:23.901565 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:23.901565 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:23.901565 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:23.901565 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:23.901565 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:23.901565 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:23.901565 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:23.901565 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:23.901565 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:23.901565 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:23.901565 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:23.901565 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:23.901565 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:23.901565 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:23.901565 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:23.901565 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:23.903068 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 02:15:23.903068 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 02:15:23.903068 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 02:15:23.903068 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 02:15:23.903068 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 02:15:23.903068 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 02:15:23.903068 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 02:15:23.903068 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:15:23.903658 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35baac8f-c17c-4b1b-b0cd-8c832386b1a7 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:23.903658 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35baac8f-c17c-4b1b-b0cd-8c832386b1a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:23.903658 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35baac8f-c17c-4b1b-b0cd-8c832386b1a7 None admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:23.903832 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 255/521] 10.222.0.66 () {56 vars in 1111 bytes} [Mon Jul 18 02:15:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:23.903832 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daef7e34-e150-42a0-a847-61639426f5d4 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:23.907281 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35baac8f-c17c-4b1b-b0cd-8c832386b1a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:23.907607 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35baac8f-c17c-4b1b-b0cd-8c832386b1a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:23.909016 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35baac8f-c17c-4b1b-b0cd-8c832386b1a7 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:23.921952 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 267/522] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 18 02:15:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:23.923069 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35baac8f-c17c-4b1b-b0cd-8c832386b1a7 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:23.964493 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:23.989635 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:24.021003 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daef7e34-e150-42a0-a847-61639426f5d4 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:24.026719 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35baac8f-c17c-4b1b-b0cd-8c832386b1a7 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:24.056641 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ca008b-572a-48a8-9ab0-11d5b7cb018c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:24.057458 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ca008b-572a-48a8-9ab0-11d5b7cb018c None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:24.057726 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ca008b-572a-48a8-9ab0-11d5b7cb018c None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:24.058001 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ca008b-572a-48a8-9ab0-11d5b7cb018c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:24.064780 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0548d2-06aa-4901-8d73-9cc3df7da5f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:24.065087 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ca008b-572a-48a8-9ab0-11d5b7cb018c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:24.065799 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0548d2-06aa-4901-8d73-9cc3df7da5f5 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:24.066259 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0548d2-06aa-4901-8d73-9cc3df7da5f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:24.066613 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ca008b-572a-48a8-9ab0-11d5b7cb018c None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:24.066827 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0548d2-06aa-4901-8d73-9cc3df7da5f5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:24.081508 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-b6ca008b-572a-48a8-9ab0-11d5b7cb018c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 02:15:24.081508 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:24.081508 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:24.081508 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:24.081508 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:24.081508 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:24.081508 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:24.081508 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:24.081508 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:24.081508 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:24.081508 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:24.081508 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:24.081508 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 02:15:24.081508 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 02:15:24.081508 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 02:15:24.082595 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 02:15:24.082595 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:24.082595 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:24.082595 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:24.082595 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:24.082595 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:24.082595 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:24.082595 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:24.082595 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:24.082595 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:24.082595 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:24.082595 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:24.082595 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:24.082595 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:24.082595 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:24.082595 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:24.082595 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:24.083844 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 02:15:24.083844 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 02:15:24.083844 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 02:15:24.083844 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 02:15:24.083844 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 02:15:24.083844 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 02:15:24.083844 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 02:15:24.083844 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:15:24.087309 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 256/523] 10.222.0.66 () {56 vars in 1111 bytes} [Mon Jul 18 02:15:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:24.088129 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ca008b-572a-48a8-9ab0-11d5b7cb018c None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:24.145937 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0548d2-06aa-4901-8d73-9cc3df7da5f5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:24.147448 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0548d2-06aa-4901-8d73-9cc3df7da5f5 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:24.188661 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:24.200058 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ca008b-572a-48a8-9ab0-11d5b7cb018c None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:24.233484 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:24.238371 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa894b5d-7d2c-4c32-a4b9-f6b7ca1cf12b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:24.239089 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa894b5d-7d2c-4c32-a4b9-f6b7ca1cf12b None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:24.239377 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa894b5d-7d2c-4c32-a4b9-f6b7ca1cf12b None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:24.239653 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa894b5d-7d2c-4c32-a4b9-f6b7ca1cf12b None admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:24.240436 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa894b5d-7d2c-4c32-a4b9-f6b7ca1cf12b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:24.240745 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa894b5d-7d2c-4c32-a4b9-f6b7ca1cf12b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:24.242218 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa894b5d-7d2c-4c32-a4b9-f6b7ca1cf12b None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:24.259508 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 268/524] 10.222.0.66 () {58 vars in 1282 bytes} [Mon Jul 18 02:15:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 02:15:24.259508 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0548d2-06aa-4901-8d73-9cc3df7da5f5 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:24.268760 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 257/525] 10.222.0.66 () {56 vars in 1148 bytes} [Mon Jul 18 02:15:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:24.268760 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa894b5d-7d2c-4c32-a4b9-f6b7ca1cf12b None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:24.368056 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0548d2-06aa-4901-8d73-9cc3df7da5f5 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:24.375710 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa894b5d-7d2c-4c32-a4b9-f6b7ca1cf12b None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:24.407965 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41a342a-97ba-4ed4-9fa6-1d4c11f5408e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:24.408714 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41a342a-97ba-4ed4-9fa6-1d4c11f5408e None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:24.409036 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41a342a-97ba-4ed4-9fa6-1d4c11f5408e None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:24.409316 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41a342a-97ba-4ed4-9fa6-1d4c11f5408e None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:24.416283 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41a342a-97ba-4ed4-9fa6-1d4c11f5408e None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:24.416533 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a510b3-ca60-496b-b7be-38a8a7c82f15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:24.417189 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a510b3-ca60-496b-b7be-38a8a7c82f15 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:24.417474 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a510b3-ca60-496b-b7be-38a8a7c82f15 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:24.417727 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41a342a-97ba-4ed4-9fa6-1d4c11f5408e None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:24.417821 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a510b3-ca60-496b-b7be-38a8a7c82f15 None admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:24.418551 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a510b3-ca60-496b-b7be-38a8a7c82f15 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:24.418888 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a510b3-ca60-496b-b7be-38a8a7c82f15 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:24.419634 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:24.420716 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a510b3-ca60-496b-b7be-38a8a7c82f15 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:24.433104 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 258/526] 10.222.0.66 () {56 vars in 1148 bytes} [Mon Jul 18 02:15:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:24.433774 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-c41a342a-97ba-4ed4-9fa6-1d4c11f5408e None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 18 02:15:24.433774 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:24.433774 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:24.433774 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:24.433774 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:24.433774 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:24.433774 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:24.433774 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:24.433774 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:24.433774 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:24.433774 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:24.433774 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:24.433774 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 02:15:24.433774 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 02:15:24.433774 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 02:15:24.433774 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 02:15:24.434763 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:24.434763 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:24.434763 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 02:15:24.434763 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 02:15:24.434763 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 02:15:24.434763 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 02:15:24.434763 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:24.434763 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:24.434763 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:24.434763 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:24.434763 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:24.434763 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:24.434763 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:24.434763 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:24.434763 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:24.434763 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:24.434763 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:24.434763 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:24.436347 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:24.436347 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:24.436347 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 02:15:24.436347 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 02:15:24.436347 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 02:15:24.436347 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 02:15:24.436347 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 02:15:24.436347 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 02:15:24.436347 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 18 02:15:24.436347 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:15:24.436951 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a510b3-ca60-496b-b7be-38a8a7c82f15 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:24.438875 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 269/527] 10.222.0.66 () {56 vars in 1102 bytes} [Mon Jul 18 02:15:24 2022] GET /identity/v3/users/neutron => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:24.439844 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41a342a-97ba-4ed4-9fa6-1d4c11f5408e None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:24.500534 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:24.542540 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41a342a-97ba-4ed4-9fa6-1d4c11f5408e None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:24.553023 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a510b3-ca60-496b-b7be-38a8a7c82f15 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:24.580054 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a4a374-7e34-4af5-970e-8b89f221a364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:24.580708 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a4a374-7e34-4af5-970e-8b89f221a364 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:24.580992 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a4a374-7e34-4af5-970e-8b89f221a364 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:24.581270 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a4a374-7e34-4af5-970e-8b89f221a364 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:24.582112 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a4a374-7e34-4af5-970e-8b89f221a364 None admin] RBAC: Adding query filter params (role.id=eea94daf3312421cb12aa8abfa73160e, scope.project.id=76f7c0ae36ae49b09dcae4c3e9a875f4, user.id=e1391b81e7234bc18e22b9e84f231256) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:24.582434 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a4a374-7e34-4af5-970e-8b89f221a364 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:24.583881 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a4a374-7e34-4af5-970e-8b89f221a364 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:24.586286 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13249c7-7efa-4f30-946d-a61041c0b379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:24.587078 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13249c7-7efa-4f30-946d-a61041c0b379 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:24.587357 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13249c7-7efa-4f30-946d-a61041c0b379 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:24.587630 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13249c7-7efa-4f30-946d-a61041c0b379 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:24.598116 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 270/528] 10.222.0.66 () {56 vars in 1374 bytes} [Mon Jul 18 02:15:24 2022] GET /identity/v3/role_assignments?user.id=e1391b81e7234bc18e22b9e84f231256&scope.project.id=76f7c0ae36ae49b09dcae4c3e9a875f4&role.id=eea94daf3312421cb12aa8abfa73160e => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:24.598116 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a4a374-7e34-4af5-970e-8b89f221a364 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:24.610633 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13249c7-7efa-4f30-946d-a61041c0b379 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:24.614892 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13249c7-7efa-4f30-946d-a61041c0b379 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:24.637728 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-c13249c7-7efa-4f30-946d-a61041c0b379 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:15:24.637728 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:24.637728 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:24.637728 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:24.637728 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:24.637728 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:24.637728 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:24.637728 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:24.637728 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:24.637728 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:24.637728 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:24.637728 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:24.637728 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 02:15:24.637728 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 02:15:24.637728 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 02:15:24.639396 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 02:15:24.639396 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:24.639396 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:24.639396 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:24.639396 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:24.639396 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:24.639396 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:24.639396 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:24.639396 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:24.639396 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:24.639396 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:24.639396 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:24.639396 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:24.639396 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:24.639396 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:24.639396 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:24.639396 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:24.640952 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 02:15:24.640952 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 02:15:24.640952 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:15:24.640952 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:15:24.640952 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 259/529] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 18 02:15:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:24.642452 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13249c7-7efa-4f30-946d-a61041c0b379 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:24.684834 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:24.741156 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a4a374-7e34-4af5-970e-8b89f221a364 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:24.784538 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f227c2c-6bb0-4bf8-ac09-3554f94d9279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:24.785126 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f227c2c-6bb0-4bf8-ac09-3554f94d9279 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:24.785408 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f227c2c-6bb0-4bf8-ac09-3554f94d9279 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:24.785687 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f227c2c-6bb0-4bf8-ac09-3554f94d9279 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:24.786544 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f227c2c-6bb0-4bf8-ac09-3554f94d9279 None admin] RBAC: Adding query filter params (role.id=eea94daf3312421cb12aa8abfa73160e, scope.project.id=76f7c0ae36ae49b09dcae4c3e9a875f4, user.id=80b138831bc54c7b986d50092d62d4af) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:24.786857 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f227c2c-6bb0-4bf8-ac09-3554f94d9279 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:24.788314 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f227c2c-6bb0-4bf8-ac09-3554f94d9279 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:24.794022 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13249c7-7efa-4f30-946d-a61041c0b379 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:24.797493 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:24.802423 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 271/530] 10.222.0.66 () {56 vars in 1374 bytes} [Mon Jul 18 02:15:24 2022] GET /identity/v3/role_assignments?user.id=80b138831bc54c7b986d50092d62d4af&scope.project.id=76f7c0ae36ae49b09dcae4c3e9a875f4&role.id=eea94daf3312421cb12aa8abfa73160e => generated 259 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:24.803456 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f227c2c-6bb0-4bf8-ac09-3554f94d9279 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:24.840909 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18915ce5-5dd6-4b3e-9214-d962df38ebb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:24.841611 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18915ce5-5dd6-4b3e-9214-d962df38ebb2 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:24.841970 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18915ce5-5dd6-4b3e-9214-d962df38ebb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:24.842288 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18915ce5-5dd6-4b3e-9214-d962df38ebb2 None admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:24.843406 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18915ce5-5dd6-4b3e-9214-d962df38ebb2 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:24.843721 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18915ce5-5dd6-4b3e-9214-d962df38ebb2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:24.845244 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18915ce5-5dd6-4b3e-9214-d962df38ebb2 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:24.889214 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 260/531] 10.222.0.66 () {56 vars in 1139 bytes} [Mon Jul 18 02:15:24 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:24.896741 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18915ce5-5dd6-4b3e-9214-d962df38ebb2 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:25.026044 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18915ce5-5dd6-4b3e-9214-d962df38ebb2 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:25.059131 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f227c2c-6bb0-4bf8-ac09-3554f94d9279 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:25.070765 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419dca23-aec4-40eb-9d28-4bad16edb87a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:25.071436 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419dca23-aec4-40eb-9d28-4bad16edb87a None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:25.072068 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419dca23-aec4-40eb-9d28-4bad16edb87a None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:25.072338 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419dca23-aec4-40eb-9d28-4bad16edb87a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:25.101468 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:25.110522 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:25.112565 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e3e827-8c13-4369-988d-393270f09dca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:25.114914 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e3e827-8c13-4369-988d-393270f09dca None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:25.114914 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e3e827-8c13-4369-988d-393270f09dca None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:25.114914 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e3e827-8c13-4369-988d-393270f09dca None admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:25.121853 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e3e827-8c13-4369-988d-393270f09dca None admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:25.122289 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e3e827-8c13-4369-988d-393270f09dca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:25.123802 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e3e827-8c13-4369-988d-393270f09dca None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:25.155053 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419dca23-aec4-40eb-9d28-4bad16edb87a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:25.155943 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 272/532] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 18 02:15:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:25.157036 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419dca23-aec4-40eb-9d28-4bad16edb87a None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:25.162223 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e3e827-8c13-4369-988d-393270f09dca None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:25.274279 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 261/533] 10.222.0.66 () {58 vars in 1282 bytes} [Mon Jul 18 02:15:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 02:15:25.279750 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419dca23-aec4-40eb-9d28-4bad16edb87a None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:25.353298 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:25.385053 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e3e827-8c13-4369-988d-393270f09dca None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:25.417822 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419dca23-aec4-40eb-9d28-4bad16edb87a None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:25.462996 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df731f9-0cc5-4f7c-bb25-d22e212e066a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:25.463323 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d26ea7-c202-4ae4-9e4f-ba149519b68a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:25.464129 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d26ea7-c202-4ae4-9e4f-ba149519b68a None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:25.464413 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d26ea7-c202-4ae4-9e4f-ba149519b68a None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:25.464687 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d26ea7-c202-4ae4-9e4f-ba149519b68a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:25.464845 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df731f9-0cc5-4f7c-bb25-d22e212e066a None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:25.465485 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df731f9-0cc5-4f7c-bb25-d22e212e066a None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:25.465936 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df731f9-0cc5-4f7c-bb25-d22e212e066a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:25.479267 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d26ea7-c202-4ae4-9e4f-ba149519b68a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:25.479447 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df731f9-0cc5-4f7c-bb25-d22e212e066a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:25.480738 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d26ea7-c202-4ae4-9e4f-ba149519b68a None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:25.481457 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df731f9-0cc5-4f7c-bb25-d22e212e066a None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:25.500961 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-3df731f9-0cc5-4f7c-bb25-d22e212e066a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:15:25.500961 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:25.500961 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:25.500961 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:25.500961 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:25.500961 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:25.500961 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:25.500961 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:25.500961 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:25.500961 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:25.500961 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:25.500961 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:25.500961 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 02:15:25.500961 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 02:15:25.500961 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 02:15:25.501880 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 02:15:25.501880 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:25.501880 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:25.501880 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:25.501880 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:25.501880 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:25.501880 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:25.501880 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:25.501880 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:25.501880 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:25.501880 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:25.501880 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:25.501880 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:25.501880 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:25.501880 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:25.501880 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:25.501880 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:25.502806 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 02:15:25.502806 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 02:15:25.502806 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:15:25.502806 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:15:25.505040 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 262/534] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 18 02:15:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 02:15:25.508259 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-38d26ea7-c202-4ae4-9e4f-ba149519b68a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 02:15:25.508259 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:25.508259 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:25.508259 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:25.508259 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:25.508259 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:25.508259 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:25.508259 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:25.508259 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:25.508259 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:25.508259 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:25.508259 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:25.508259 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 02:15:25.508259 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 02:15:25.508259 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 02:15:25.509084 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 02:15:25.509084 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:25.509084 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:25.509084 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:25.509084 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:25.509084 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:25.509084 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:25.509084 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:25.509084 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:25.509084 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:25.509084 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:25.509084 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:25.509084 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:25.509084 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:25.509084 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:25.509084 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:25.509084 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:25.509974 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 02:15:25.509974 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 02:15:25.509974 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 02:15:25.509974 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 02:15:25.509974 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 02:15:25.509974 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 02:15:25.509974 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 02:15:25.509974 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:15:25.511811 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df731f9-0cc5-4f7c-bb25-d22e212e066a None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:25.514749 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 273/535] 10.222.0.66 () {56 vars in 1111 bytes} [Mon Jul 18 02:15:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 353 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:25.523057 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d26ea7-c202-4ae4-9e4f-ba149519b68a None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:25.603528 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:25.663473 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d26ea7-c202-4ae4-9e4f-ba149519b68a None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:25.700415 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278d7429-5a48-4bc9-a8a0-c02ba2b06f4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:25.701996 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278d7429-5a48-4bc9-a8a0-c02ba2b06f4c None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:25.702526 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278d7429-5a48-4bc9-a8a0-c02ba2b06f4c None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:25.703001 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278d7429-5a48-4bc9-a8a0-c02ba2b06f4c None admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:25.703990 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278d7429-5a48-4bc9-a8a0-c02ba2b06f4c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:25.704538 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278d7429-5a48-4bc9-a8a0-c02ba2b06f4c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:25.706219 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278d7429-5a48-4bc9-a8a0-c02ba2b06f4c None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:25.725879 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 274/536] 10.222.0.66 () {56 vars in 1148 bytes} [Mon Jul 18 02:15:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:25.727568 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68d1935-74f9-46ad-aa6a-4e867118bf46 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:25.728063 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68d1935-74f9-46ad-aa6a-4e867118bf46 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:25.728568 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68d1935-74f9-46ad-aa6a-4e867118bf46 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:25.730322 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 275/537] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:15:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 02:15:25.731314 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68d1935-74f9-46ad-aa6a-4e867118bf46 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:25.742842 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df731f9-0cc5-4f7c-bb25-d22e212e066a None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:25.766488 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:25.777933 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b084290-d95b-45a7-b1bb-6e535fba9462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:25.778793 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b084290-d95b-45a7-b1bb-6e535fba9462 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:25.778910 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b084290-d95b-45a7-b1bb-6e535fba9462 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:25.779772 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b084290-d95b-45a7-b1bb-6e535fba9462 None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:25.780493 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b084290-d95b-45a7-b1bb-6e535fba9462 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:25.788445 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b084290-d95b-45a7-b1bb-6e535fba9462 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:25.788445 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b084290-d95b-45a7-b1bb-6e535fba9462 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:25.794374 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 263/538] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 18 02:15:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:25.795853 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503a406d-ea42-42cf-b7ee-b2f8c2ceb156 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:25.796142 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503a406d-ea42-42cf-b7ee-b2f8c2ceb156 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:25.796416 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503a406d-ea42-42cf-b7ee-b2f8c2ceb156 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:25.813271 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-503a406d-ea42-42cf-b7ee-b2f8c2ceb156 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:15:25.814484 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-503a406d-ea42-42cf-b7ee-b2f8c2ceb156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:15:25.844474 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68d1935-74f9-46ad-aa6a-4e867118bf46 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:25.852677 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:25.882539 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef017f18-d21c-4e02-b4a9-cd7183137629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:25.883533 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef017f18-d21c-4e02-b4a9-cd7183137629 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:25.884035 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef017f18-d21c-4e02-b4a9-cd7183137629 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:25.884529 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef017f18-d21c-4e02-b4a9-cd7183137629 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:25.885556 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef017f18-d21c-4e02-b4a9-cd7183137629 None admin] RBAC: Adding query filter params (role.id=eea94daf3312421cb12aa8abfa73160e, scope.project.id=76f7c0ae36ae49b09dcae4c3e9a875f4, user.id=cd49388facdf446885e55b45ef91f01e) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:25.886071 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef017f18-d21c-4e02-b4a9-cd7183137629 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:25.887773 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef017f18-d21c-4e02-b4a9-cd7183137629 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:25.890326 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 264/539] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:15:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 02:15:25.891229 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503a406d-ea42-42cf-b7ee-b2f8c2ceb156 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:25.894914 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 276/540] 10.222.0.66 () {56 vars in 1374 bytes} [Mon Jul 18 02:15:25 2022] GET /identity/v3/role_assignments?user.id=cd49388facdf446885e55b45ef91f01e&scope.project.id=76f7c0ae36ae49b09dcae4c3e9a875f4&role.id=eea94daf3312421cb12aa8abfa73160e => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 02:15:25.913840 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef017f18-d21c-4e02-b4a9-cd7183137629 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:26.015645 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503a406d-ea42-42cf-b7ee-b2f8c2ceb156 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:26.018466 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:26.055599 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7370ec3-509e-44b0-9d37-97c7c044011d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:26.056408 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7370ec3-509e-44b0-9d37-97c7c044011d None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:26.056686 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7370ec3-509e-44b0-9d37-97c7c044011d None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:26.056959 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7370ec3-509e-44b0-9d37-97c7c044011d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:26.064936 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7370ec3-509e-44b0-9d37-97c7c044011d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:26.066459 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7370ec3-509e-44b0-9d37-97c7c044011d None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:26.081792 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-c7370ec3-509e-44b0-9d37-97c7c044011d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 02:15:26.081792 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:26.081792 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:26.081792 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:26.081792 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:26.081792 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:26.081792 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:26.081792 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:26.081792 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:26.081792 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:26.081792 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:26.081792 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:26.081792 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 02:15:26.081792 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 02:15:26.081792 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 02:15:26.082816 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 02:15:26.082816 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:26.082816 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:26.082816 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:26.082816 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:26.082816 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:26.082816 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:26.082816 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:26.082816 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:26.082816 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:26.082816 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:26.082816 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:26.082816 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:26.082816 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:26.082816 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:26.082816 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:26.082816 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:26.083678 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 02:15:26.083678 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 02:15:26.083678 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 02:15:26.083678 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 02:15:26.083678 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 02:15:26.083678 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 02:15:26.083678 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 02:15:26.083678 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:15:26.083678 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 265/541] 10.222.0.66 () {56 vars in 1111 bytes} [Mon Jul 18 02:15:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:26.091301 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7370ec3-509e-44b0-9d37-97c7c044011d None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:26.115231 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef017f18-d21c-4e02-b4a9-cd7183137629 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:26.162306 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7648aca4-baba-4510-9db0-d3c893f40238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:26.163198 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7648aca4-baba-4510-9db0-d3c893f40238 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:26.163485 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7648aca4-baba-4510-9db0-d3c893f40238 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:26.163760 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7648aca4-baba-4510-9db0-d3c893f40238 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:26.180395 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7648aca4-baba-4510-9db0-d3c893f40238 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 02:15:26.191555 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-7648aca4-baba-4510-9db0-d3c893f40238 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 02:15:26.191555 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:26.191555 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:26.191555 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:26.191555 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:26.191555 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:26.191555 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:26.191555 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:26.191555 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:26.191555 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:26.191555 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:26.191555 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:26.191555 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 02:15:26.191555 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 02:15:26.191555 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 02:15:26.191555 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 02:15:26.192492 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 02:15:26.192492 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 02:15:26.192492 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 02:15:26.192492 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 02:15:26.192492 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:26.192492 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:26.192492 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:26.192492 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:26.192492 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:26.192492 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:26.192492 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:26.192492 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:26.192492 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:26.192492 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:26.192492 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:26.192492 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:26.193310 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:26.193310 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:26.193310 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:26.193310 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:26.193310 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 02:15:26.193310 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 02:15:26.193310 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 02:15:26.193310 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 02:15:26.193310 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 02:15:26.193310 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 02:15:26.193310 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 02:15:26.193310 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:15:26.194433 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 277/542] 10.222.0.66 () {56 vars in 1102 bytes} [Mon Jul 18 02:15:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 02:15:26.203076 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7648aca4-baba-4510-9db0-d3c893f40238 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:26.226424 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7370ec3-509e-44b0-9d37-97c7c044011d None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:26.259742 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:26.291674 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0962b732-558e-40cb-a896-4dd05d613f9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:26.292336 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0962b732-558e-40cb-a896-4dd05d613f9c None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:26.292613 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0962b732-558e-40cb-a896-4dd05d613f9c None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:26.292885 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0962b732-558e-40cb-a896-4dd05d613f9c None admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:26.293663 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0962b732-558e-40cb-a896-4dd05d613f9c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:26.294026 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0962b732-558e-40cb-a896-4dd05d613f9c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:26.296070 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0962b732-558e-40cb-a896-4dd05d613f9c None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:26.320362 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 266/543] 10.222.0.66 () {56 vars in 1148 bytes} [Mon Jul 18 02:15:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:26.326366 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0962b732-558e-40cb-a896-4dd05d613f9c None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:26.362463 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:26.382977 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7648aca4-baba-4510-9db0-d3c893f40238 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:26.448813 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e36e51-cf86-4b17-8ea9-2c772ed68c34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:26.449396 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e36e51-cf86-4b17-8ea9-2c772ed68c34 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:26.449678 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e36e51-cf86-4b17-8ea9-2c772ed68c34 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:26.449957 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e36e51-cf86-4b17-8ea9-2c772ed68c34 None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:26.450742 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e36e51-cf86-4b17-8ea9-2c772ed68c34 None admin] RBAC: Adding query filter params (name=service) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:26.451570 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e36e51-cf86-4b17-8ea9-2c772ed68c34 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:26.453055 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e36e51-cf86-4b17-8ea9-2c772ed68c34 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:26.462282 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 278/544] 10.222.0.66 () {56 vars in 1103 bytes} [Mon Jul 18 02:15:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:26.468431 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e36e51-cf86-4b17-8ea9-2c772ed68c34 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:26.503031 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0962b732-558e-40cb-a896-4dd05d613f9c None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:26.554000 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd464ff-388c-4ff2-8526-b5a3dba17b58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:26.554694 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd464ff-388c-4ff2-8526-b5a3dba17b58 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:26.554976 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd464ff-388c-4ff2-8526-b5a3dba17b58 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:26.555251 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd464ff-388c-4ff2-8526-b5a3dba17b58 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:26.556078 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd464ff-388c-4ff2-8526-b5a3dba17b58 None admin] RBAC: Adding query filter params (role.id=eea94daf3312421cb12aa8abfa73160e, scope.project.id=76f7c0ae36ae49b09dcae4c3e9a875f4, user.id=eaf19a259c2741cea375a38763e33cdc) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:26.556387 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd464ff-388c-4ff2-8526-b5a3dba17b58 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:26.557835 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd464ff-388c-4ff2-8526-b5a3dba17b58 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:26.564471 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 267/545] 10.222.0.66 () {56 vars in 1374 bytes} [Mon Jul 18 02:15:26 2022] GET /identity/v3/role_assignments?user.id=eaf19a259c2741cea375a38763e33cdc&scope.project.id=76f7c0ae36ae49b09dcae4c3e9a875f4&role.id=eea94daf3312421cb12aa8abfa73160e => generated 259 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:26.642480 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:26.651994 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e36e51-cf86-4b17-8ea9-2c772ed68c34 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:26.688847 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a75ed50-e57a-484a-a9e7-82bcf4f3405e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:26.689664 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a75ed50-e57a-484a-a9e7-82bcf4f3405e None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:26.689953 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a75ed50-e57a-484a-a9e7-82bcf4f3405e None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:26.690273 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a75ed50-e57a-484a-a9e7-82bcf4f3405e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:26.703360 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a75ed50-e57a-484a-a9e7-82bcf4f3405e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:26.704747 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a75ed50-e57a-484a-a9e7-82bcf4f3405e None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:26.723069 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-9a75ed50-e57a-484a-a9e7-82bcf4f3405e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:15:26.723069 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:26.723069 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:26.723069 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:26.723069 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:26.723069 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:26.723069 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:26.723069 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:26.723069 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:26.723069 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:26.723069 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:26.723069 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:26.723069 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 02:15:26.723069 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 02:15:26.723069 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 02:15:26.724011 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 02:15:26.724011 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:26.724011 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:26.724011 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:26.724011 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:26.724011 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:26.724011 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:26.724011 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:26.724011 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:26.724011 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:26.724011 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:26.724011 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:26.724011 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:26.724011 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:26.724011 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:26.724011 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:26.724011 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:26.724973 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 02:15:26.724973 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 02:15:26.724973 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:15:26.724973 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:15:26.724973 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 279/546] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 18 02:15:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:26.731414 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd464ff-388c-4ff2-8526-b5a3dba17b58 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:26.845126 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:26.858320 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd464ff-388c-4ff2-8526-b5a3dba17b58 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:26.895115 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a538e7-5b5b-46e2-9d9f-e33ee88ece78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:26.895782 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a538e7-5b5b-46e2-9d9f-e33ee88ece78 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:26.896066 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a538e7-5b5b-46e2-9d9f-e33ee88ece78 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:26.896341 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a538e7-5b5b-46e2-9d9f-e33ee88ece78 None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:26.897060 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a538e7-5b5b-46e2-9d9f-e33ee88ece78 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:26.897367 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a538e7-5b5b-46e2-9d9f-e33ee88ece78 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:26.898772 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a538e7-5b5b-46e2-9d9f-e33ee88ece78 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:26.911395 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 268/547] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 18 02:15:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:26.916700 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a75ed50-e57a-484a-a9e7-82bcf4f3405e None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:26.989570 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bd5039-af11-4b70-aa5e-e69bf44478f1 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:26.990190 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bd5039-af11-4b70-aa5e-e69bf44478f1 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:26.990190 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bd5039-af11-4b70-aa5e-e69bf44478f1 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:26.991420 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 269/548] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:15:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 02:15:26.999833 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9febc23-28b1-423a-8bac-be4c26afff87 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:27.000117 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9febc23-28b1-423a-8bac-be4c26afff87 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:27.000396 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9febc23-28b1-423a-8bac-be4c26afff87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:27.031028 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a75ed50-e57a-484a-a9e7-82bcf4f3405e None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:27.039108 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-a9febc23-28b1-423a-8bac-be4c26afff87 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:15:27.040320 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-a9febc23-28b1-423a-8bac-be4c26afff87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:15:27.068744 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5546a6-c3f7-40d0-a354-4cdd014a21a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:27.069984 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5546a6-c3f7-40d0-a354-4cdd014a21a7 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:27.070326 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5546a6-c3f7-40d0-a354-4cdd014a21a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:27.070646 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5546a6-c3f7-40d0-a354-4cdd014a21a7 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:27.080809 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5546a6-c3f7-40d0-a354-4cdd014a21a7 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:27.082944 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5546a6-c3f7-40d0-a354-4cdd014a21a7 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:27.099313 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-ed5546a6-c3f7-40d0-a354-4cdd014a21a7 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 18 02:15:27.099313 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:27.099313 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:27.099313 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:27.099313 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:27.099313 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:27.099313 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:27.099313 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:27.099313 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:27.099313 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:27.099313 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:27.099313 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:27.099313 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 02:15:27.099313 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 02:15:27.099313 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 02:15:27.099313 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 02:15:27.100394 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:27.100394 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:27.100394 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 02:15:27.100394 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 02:15:27.100394 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 02:15:27.100394 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 02:15:27.100394 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:27.100394 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:27.100394 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:27.100394 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:27.100394 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:27.100394 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:27.100394 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:27.100394 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:27.100394 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:27.100394 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:27.100394 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:27.100394 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:27.101331 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:27.101331 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:27.101331 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 02:15:27.101331 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 02:15:27.101331 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 02:15:27.101331 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 02:15:27.101331 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 02:15:27.101331 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 02:15:27.101331 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 18 02:15:27.101331 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:15:27.105365 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 280/549] 10.222.0.66 () {56 vars in 1096 bytes} [Mon Jul 18 02:15:26 2022] GET /identity/v3/users/swift => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 02:15:27.107083 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5546a6-c3f7-40d0-a354-4cdd014a21a7 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:27.122453 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:27.221822 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5546a6-c3f7-40d0-a354-4cdd014a21a7 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:27.227552 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 270/550] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:15:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 02:15:27.229109 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c78d8b-4156-4111-a2b5-ebd73a10fc28 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:27.229607 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c78d8b-4156-4111-a2b5-ebd73a10fc28 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:27.230141 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c78d8b-4156-4111-a2b5-ebd73a10fc28 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:27.236524 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 271/551] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:15:27 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:15:27.237937 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22de0e72-a30a-45a5-b729-79d47cc4e613 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:27.238455 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22de0e72-a30a-45a5-b729-79d47cc4e613 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:27.238956 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22de0e72-a30a-45a5-b729-79d47cc4e613 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:27.262276 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-22de0e72-a30a-45a5-b729-79d47cc4e613 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:15:27.262276 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-22de0e72-a30a-45a5-b729-79d47cc4e613 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:15:27.271894 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d505b1af-5ca5-4843-aff4-b4250af7fffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:27.272580 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d505b1af-5ca5-4843-aff4-b4250af7fffe None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:27.272884 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d505b1af-5ca5-4843-aff4-b4250af7fffe None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:27.273586 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d505b1af-5ca5-4843-aff4-b4250af7fffe None admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:27.274496 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d505b1af-5ca5-4843-aff4-b4250af7fffe None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:27.275183 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d505b1af-5ca5-4843-aff4-b4250af7fffe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:27.276929 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d505b1af-5ca5-4843-aff4-b4250af7fffe None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:27.302070 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 281/552] 10.222.0.66 () {56 vars in 1135 bytes} [Mon Jul 18 02:15:27 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:27.302937 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d505b1af-5ca5-4843-aff4-b4250af7fffe None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:27.353818 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 272/553] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:15:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 02:15:27.355457 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22de0e72-a30a-45a5-b729-79d47cc4e613 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:27.369130 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:27.390767 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:27.431259 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d505b1af-5ca5-4843-aff4-b4250af7fffe None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:27.468475 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31f5bf3-c3b7-454d-9bc4-a55009443775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:27.469402 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31f5bf3-c3b7-454d-9bc4-a55009443775 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:27.469799 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31f5bf3-c3b7-454d-9bc4-a55009443775 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:27.470605 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31f5bf3-c3b7-454d-9bc4-a55009443775 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:27.481084 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31f5bf3-c3b7-454d-9bc4-a55009443775 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 02:15:27.484126 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22de0e72-a30a-45a5-b729-79d47cc4e613 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:27.489972 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-a31f5bf3-c3b7-454d-9bc4-a55009443775 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 02:15:27.489972 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:27.489972 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:27.489972 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:27.489972 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:27.489972 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:27.489972 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:27.489972 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:27.489972 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:27.489972 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:27.489972 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:27.489972 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:27.489972 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 02:15:27.489972 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 02:15:27.489972 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 02:15:27.489972 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 02:15:27.491169 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 02:15:27.491169 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 02:15:27.491169 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 02:15:27.491169 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 02:15:27.491169 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:27.491169 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:27.491169 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:27.491169 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:27.491169 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:27.491169 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:27.491169 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:27.491169 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:27.491169 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:27.491169 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:27.491169 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:27.491169 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:27.492175 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:27.492175 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:27.492175 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:27.492175 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:27.492175 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 02:15:27.492175 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 02:15:27.492175 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 02:15:27.492175 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 02:15:27.492175 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 02:15:27.492175 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 02:15:27.492175 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 02:15:27.492175 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:15:27.500026 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 282/554] 10.222.0.66 () {56 vars in 1102 bytes} [Mon Jul 18 02:15:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:27.501054 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31f5bf3-c3b7-454d-9bc4-a55009443775 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:27.528108 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c632940e-dd5f-42ca-b466-70e755d1daf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:27.529948 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c632940e-dd5f-42ca-b466-70e755d1daf2 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:27.530496 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c632940e-dd5f-42ca-b466-70e755d1daf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:27.530953 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c632940e-dd5f-42ca-b466-70e755d1daf2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:27.606798 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c632940e-dd5f-42ca-b466-70e755d1daf2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:27.609172 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31f5bf3-c3b7-454d-9bc4-a55009443775 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:27.609293 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c632940e-dd5f-42ca-b466-70e755d1daf2 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:27.620756 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:27.632845 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:27.645029 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43400616-c774-44d4-88c5-d55ed3704fde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:27.646001 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43400616-c774-44d4-88c5-d55ed3704fde None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:27.646185 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43400616-c774-44d4-88c5-d55ed3704fde None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:27.646601 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43400616-c774-44d4-88c5-d55ed3704fde None admin] PATH_INFO: `/v3/roles/service` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:27.658009 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43400616-c774-44d4-88c5-d55ed3704fde None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 02:15:27.665955 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-43400616-c774-44d4-88c5-d55ed3704fde None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 02:15:27.665955 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:27.665955 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:27.665955 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:27.665955 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:27.665955 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:27.665955 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:27.665955 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:27.665955 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:27.665955 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:27.665955 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:27.665955 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:27.665955 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 02:15:27.665955 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 02:15:27.665955 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 02:15:27.665955 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 02:15:27.666916 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 02:15:27.666916 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 02:15:27.666916 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 02:15:27.666916 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 02:15:27.666916 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:27.666916 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:27.666916 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:27.666916 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:27.666916 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:27.666916 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:27.666916 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:27.666916 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:27.666916 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:27.666916 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:27.666916 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:27.666916 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:27.667753 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:27.667753 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:27.667753 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:27.667753 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:27.667753 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 02:15:27.667753 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 02:15:27.667753 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 02:15:27.667753 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 02:15:27.667753 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 02:15:27.667753 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 02:15:27.667753 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 02:15:27.667753 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:15:27.672441 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 283/555] 10.222.0.66 () {56 vars in 1102 bytes} [Mon Jul 18 02:15:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:27.673757 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43400616-c774-44d4-88c5-d55ed3704fde None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:27.736249 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 273/556] 10.222.0.66 () {58 vars in 1282 bytes} [Mon Jul 18 02:15:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 02:15:27.737511 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c632940e-dd5f-42ca-b466-70e755d1daf2 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:27.774907 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43400616-c774-44d4-88c5-d55ed3704fde None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:27.810095 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b171a072-15de-4430-9e86-b5882e332350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:27.810937 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b171a072-15de-4430-9e86-b5882e332350 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:27.811300 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b171a072-15de-4430-9e86-b5882e332350 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:27.811900 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b171a072-15de-4430-9e86-b5882e332350 None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:27.812430 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b171a072-15de-4430-9e86-b5882e332350 None admin] RBAC: Adding query filter params (name=service) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:27.812797 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b171a072-15de-4430-9e86-b5882e332350 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:27.814265 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b171a072-15de-4430-9e86-b5882e332350 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:27.823055 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 284/557] 10.222.0.66 () {56 vars in 1103 bytes} [Mon Jul 18 02:15:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:27.828840 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b171a072-15de-4430-9e86-b5882e332350 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:27.850038 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c632940e-dd5f-42ca-b466-70e755d1daf2 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:27.861356 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:27.876306 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:27.889926 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b25a35a-2d5c-4f64-8497-aa7c91fad92f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:27.890907 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b25a35a-2d5c-4f64-8497-aa7c91fad92f None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:27.891393 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b25a35a-2d5c-4f64-8497-aa7c91fad92f None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:27.891862 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b25a35a-2d5c-4f64-8497-aa7c91fad92f None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:27.892877 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b25a35a-2d5c-4f64-8497-aa7c91fad92f None admin] RBAC: Adding query filter params (name=service) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:27.893366 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b25a35a-2d5c-4f64-8497-aa7c91fad92f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:27.894963 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b25a35a-2d5c-4f64-8497-aa7c91fad92f None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:27.905300 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 274/558] 10.222.0.66 () {56 vars in 1103 bytes} [Mon Jul 18 02:15:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:27.909159 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b25a35a-2d5c-4f64-8497-aa7c91fad92f None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:27.941458 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b171a072-15de-4430-9e86-b5882e332350 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:27.977287 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b93fac-66a4-4974-bef2-a25a83103bfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:27.978204 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b93fac-66a4-4974-bef2-a25a83103bfd None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:27.978475 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b93fac-66a4-4974-bef2-a25a83103bfd None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:27.978867 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b93fac-66a4-4974-bef2-a25a83103bfd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:27.991063 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b93fac-66a4-4974-bef2-a25a83103bfd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:27.992643 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b93fac-66a4-4974-bef2-a25a83103bfd None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:28.011187 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-85b93fac-66a4-4974-bef2-a25a83103bfd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:15:28.011187 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:28.011187 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:28.011187 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:28.011187 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:28.011187 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:28.011187 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:28.011187 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:28.011187 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:28.011187 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:28.011187 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:28.011187 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:28.011187 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 02:15:28.011187 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 02:15:28.011187 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 02:15:28.012164 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 02:15:28.012164 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:28.012164 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:28.012164 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:28.012164 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:28.012164 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:28.012164 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:28.012164 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:28.012164 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:28.012164 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:28.012164 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:28.012164 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:28.012164 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:28.012164 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:28.012164 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:28.012164 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:28.012164 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:28.013060 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 02:15:28.013060 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 02:15:28.013060 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:15:28.013060 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:15:28.017786 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 285/559] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 18 02:15:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:28.018764 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b93fac-66a4-4974-bef2-a25a83103bfd None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:28.037295 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b25a35a-2d5c-4f64-8497-aa7c91fad92f None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:28.076423 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8039e0-5b6b-4800-a6c1-be4286369918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:28.077685 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8039e0-5b6b-4800-a6c1-be4286369918 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:28.078279 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8039e0-5b6b-4800-a6c1-be4286369918 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:28.078741 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8039e0-5b6b-4800-a6c1-be4286369918 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:28.090697 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8039e0-5b6b-4800-a6c1-be4286369918 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:28.092672 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8039e0-5b6b-4800-a6c1-be4286369918 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:28.111449 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-2c8039e0-5b6b-4800-a6c1-be4286369918 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:15:28.111449 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:28.111449 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:28.111449 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:28.111449 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:28.111449 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:28.111449 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:28.111449 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:28.111449 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:28.111449 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:28.111449 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:28.111449 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:28.111449 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 02:15:28.111449 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 02:15:28.111449 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 02:15:28.112437 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 02:15:28.112437 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:28.112437 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:28.112437 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:28.112437 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:28.112437 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:28.112437 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:28.112437 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:28.112437 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:28.112437 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:28.112437 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:28.112437 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:28.112437 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:28.112437 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:28.112437 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:28.112437 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:28.112437 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:28.113334 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 02:15:28.113334 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 02:15:28.113334 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:15:28.113334 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:15:28.115702 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 275/560] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 18 02:15:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:28.116816 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8039e0-5b6b-4800-a6c1-be4286369918 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:28.121949 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:28.131293 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b93fac-66a4-4974-bef2-a25a83103bfd None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:28.151778 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:28.179015 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96fc6b8-8e30-4da8-b0d8-da24284be6b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:28.180591 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96fc6b8-8e30-4da8-b0d8-da24284be6b8 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:28.181057 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96fc6b8-8e30-4da8-b0d8-da24284be6b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:28.181763 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96fc6b8-8e30-4da8-b0d8-da24284be6b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:28.197506 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96fc6b8-8e30-4da8-b0d8-da24284be6b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:28.199537 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96fc6b8-8e30-4da8-b0d8-da24284be6b8 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:28.224343 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-b96fc6b8-8e30-4da8-b0d8-da24284be6b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:15:28.224343 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:28.224343 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:28.224343 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:28.224343 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:28.224343 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:28.224343 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:28.224343 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:28.224343 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:28.224343 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:28.224343 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:28.224343 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:28.224343 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 02:15:28.224343 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 02:15:28.224343 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 02:15:28.225306 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 02:15:28.225306 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:28.225306 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:28.225306 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:28.225306 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:28.225306 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:28.225306 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:28.225306 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:28.225306 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:28.225306 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:28.225306 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:28.225306 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:28.225306 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:28.225306 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:28.225306 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:28.225306 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:28.225306 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:28.226326 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 02:15:28.226326 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 02:15:28.226326 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:15:28.226326 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:15:28.232238 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 286/561] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 18 02:15:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:28.232238 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96fc6b8-8e30-4da8-b0d8-da24284be6b8 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:28.244253 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8039e0-5b6b-4800-a6c1-be4286369918 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:28.282521 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175f5a86-c28a-4658-9098-d2f7b4ac4ee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:28.283427 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175f5a86-c28a-4658-9098-d2f7b4ac4ee2 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:28.283826 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175f5a86-c28a-4658-9098-d2f7b4ac4ee2 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:28.284238 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175f5a86-c28a-4658-9098-d2f7b4ac4ee2 None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:28.285361 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175f5a86-c28a-4658-9098-d2f7b4ac4ee2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:28.285674 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175f5a86-c28a-4658-9098-d2f7b4ac4ee2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:28.287410 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175f5a86-c28a-4658-9098-d2f7b4ac4ee2 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:28.302137 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 276/562] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 18 02:15:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:28.302855 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175f5a86-c28a-4658-9098-d2f7b4ac4ee2 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:28.372160 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96fc6b8-8e30-4da8-b0d8-da24284be6b8 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:28.402422 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:28.409697 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758652f4-80b4-4ea6-b8ed-92b1cdff336d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:28.410392 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758652f4-80b4-4ea6-b8ed-92b1cdff336d None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:28.410680 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758652f4-80b4-4ea6-b8ed-92b1cdff336d None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:28.410958 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758652f4-80b4-4ea6-b8ed-92b1cdff336d None admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:28.411682 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758652f4-80b4-4ea6-b8ed-92b1cdff336d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:28.411992 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758652f4-80b4-4ea6-b8ed-92b1cdff336d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:28.413435 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758652f4-80b4-4ea6-b8ed-92b1cdff336d None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:28.421728 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175f5a86-c28a-4658-9098-d2f7b4ac4ee2 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:28.426501 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 287/563] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 18 02:15:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:28.427224 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758652f4-80b4-4ea6-b8ed-92b1cdff336d None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:28.439003 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:28.462735 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb538ad-6bd5-4df6-969d-11edc7fbe7b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:28.463397 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb538ad-6bd5-4df6-969d-11edc7fbe7b8 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:28.463677 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb538ad-6bd5-4df6-969d-11edc7fbe7b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:28.463954 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb538ad-6bd5-4df6-969d-11edc7fbe7b8 None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:28.464673 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb538ad-6bd5-4df6-969d-11edc7fbe7b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:28.464987 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb538ad-6bd5-4df6-969d-11edc7fbe7b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:28.467119 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb538ad-6bd5-4df6-969d-11edc7fbe7b8 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:28.483987 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 277/564] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 18 02:15:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:28.485359 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d653580a-aa0e-435b-8dd1-553052e810f7 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:28.485645 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d653580a-aa0e-435b-8dd1-553052e810f7 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:28.485919 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d653580a-aa0e-435b-8dd1-553052e810f7 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:28.487367 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 278/565] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:15:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:15:28.487897 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d653580a-aa0e-435b-8dd1-553052e810f7 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:28.536918 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758652f4-80b4-4ea6-b8ed-92b1cdff336d None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:28.572048 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cea89f-aafb-4d0c-92bf-b4c270b762a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:28.573076 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cea89f-aafb-4d0c-92bf-b4c270b762a0 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:28.573507 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cea89f-aafb-4d0c-92bf-b4c270b762a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:28.573929 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cea89f-aafb-4d0c-92bf-b4c270b762a0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:28.581388 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cea89f-aafb-4d0c-92bf-b4c270b762a0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:28.583054 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cea89f-aafb-4d0c-92bf-b4c270b762a0 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:28.596850 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-20cea89f-aafb-4d0c-92bf-b4c270b762a0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 02:15:28.596850 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:28.596850 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:28.596850 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:28.596850 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:28.596850 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:28.596850 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:28.596850 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:28.596850 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:28.596850 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:28.596850 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:28.596850 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:28.596850 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 02:15:28.596850 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 02:15:28.596850 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 02:15:28.597816 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 02:15:28.597816 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:28.597816 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:28.597816 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:28.597816 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:28.597816 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:28.597816 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:28.597816 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:28.597816 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:28.597816 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:28.597816 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:28.597816 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:28.597816 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:28.597816 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:28.597816 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:28.597816 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:28.597816 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:28.602623 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 02:15:28.602623 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 02:15:28.602623 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 02:15:28.602623 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 02:15:28.602623 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 02:15:28.602623 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 02:15:28.602623 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 02:15:28.602623 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:15:28.602623 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 288/566] 10.222.0.66 () {56 vars in 1111 bytes} [Mon Jul 18 02:15:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:28.602623 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cea89f-aafb-4d0c-92bf-b4c270b762a0 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:28.604101 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d653580a-aa0e-435b-8dd1-553052e810f7 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:28.644423 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7829b6-0a87-497f-887c-4b719cdeafd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:28.645126 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7829b6-0a87-497f-887c-4b719cdeafd7 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:28.645408 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7829b6-0a87-497f-887c-4b719cdeafd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:28.645680 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7829b6-0a87-497f-887c-4b719cdeafd7 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:28.652864 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7829b6-0a87-497f-887c-4b719cdeafd7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:28.655162 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7829b6-0a87-497f-887c-4b719cdeafd7 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:28.673017 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-4d7829b6-0a87-497f-887c-4b719cdeafd7 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 18 02:15:28.673017 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:28.673017 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:28.673017 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:28.673017 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:28.673017 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:28.673017 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:28.673017 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:28.673017 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:28.673017 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:28.673017 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:28.673017 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:28.673017 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 02:15:28.673017 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 02:15:28.673017 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 02:15:28.673017 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 02:15:28.674198 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:28.674198 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:28.674198 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 02:15:28.674198 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 02:15:28.674198 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 02:15:28.674198 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 02:15:28.674198 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:28.674198 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:28.674198 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:28.674198 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:28.674198 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:28.674198 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:28.674198 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:28.674198 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:28.674198 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:28.674198 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:28.674198 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:28.674198 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:28.675589 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:28.675589 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:28.675589 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 02:15:28.675589 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 02:15:28.675589 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 02:15:28.675589 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 02:15:28.675589 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 02:15:28.675589 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 02:15:28.675589 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 18 02:15:28.675589 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:15:28.675589 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 279/567] 10.222.0.66 () {56 vars in 1099 bytes} [Mon Jul 18 02:15:28 2022] GET /identity/v3/users/glance => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:28.675589 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61448501-2133-4c7c-ae2c-5c82d76b37fb None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:28.676278 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61448501-2133-4c7c-ae2c-5c82d76b37fb None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:28.676278 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61448501-2133-4c7c-ae2c-5c82d76b37fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:28.680736 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:28.682703 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:28.694152 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-61448501-2133-4c7c-ae2c-5c82d76b37fb None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:15:28.699131 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-61448501-2133-4c7c-ae2c-5c82d76b37fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:15:28.717156 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cea89f-aafb-4d0c-92bf-b4c270b762a0 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:28.751091 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62007c98-d787-462d-a987-cdf9bb268f68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:28.751889 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62007c98-d787-462d-a987-cdf9bb268f68 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:28.752173 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62007c98-d787-462d-a987-cdf9bb268f68 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:28.752451 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62007c98-d787-462d-a987-cdf9bb268f68 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:28.760240 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62007c98-d787-462d-a987-cdf9bb268f68 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:28.761713 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62007c98-d787-462d-a987-cdf9bb268f68 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:28.766714 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 280/568] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:15:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 02:15:28.767341 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61448501-2133-4c7c-ae2c-5c82d76b37fb None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:28.790924 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-62007c98-d787-462d-a987-cdf9bb268f68 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 18 02:15:28.790924 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:28.790924 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:28.790924 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:28.790924 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:28.790924 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:28.790924 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:28.790924 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:28.790924 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:28.790924 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:28.790924 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:28.790924 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:28.790924 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 02:15:28.790924 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 02:15:28.790924 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 02:15:28.790924 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 02:15:28.792139 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:28.792139 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:28.792139 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 02:15:28.792139 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 02:15:28.792139 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 02:15:28.792139 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 02:15:28.792139 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:28.792139 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:28.792139 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:28.792139 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:28.792139 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:28.792139 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:28.792139 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:28.792139 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:28.792139 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:28.792139 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:28.792139 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:28.792139 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:28.793329 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:28.793329 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:28.793329 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 02:15:28.793329 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 02:15:28.793329 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 02:15:28.793329 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 02:15:28.793329 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 02:15:28.793329 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 02:15:28.793329 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 18 02:15:28.793329 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:15:28.793329 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 289/569] 10.222.0.66 () {56 vars in 1093 bytes} [Mon Jul 18 02:15:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:28.793329 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62007c98-d787-462d-a987-cdf9bb268f68 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:28.877717 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61448501-2133-4c7c-ae2c-5c82d76b37fb None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:28.906126 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62007c98-d787-462d-a987-cdf9bb268f68 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:28.907813 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:28.912607 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55a0379-b9bc-4dc7-a415-aba5ba18c8c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:28.913319 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55a0379-b9bc-4dc7-a415-aba5ba18c8c8 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:28.913602 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55a0379-b9bc-4dc7-a415-aba5ba18c8c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:28.913929 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55a0379-b9bc-4dc7-a415-aba5ba18c8c8 None admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:28.914750 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55a0379-b9bc-4dc7-a415-aba5ba18c8c8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:28.915057 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55a0379-b9bc-4dc7-a415-aba5ba18c8c8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:28.916496 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55a0379-b9bc-4dc7-a415-aba5ba18c8c8 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:28.928823 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 281/570] 10.222.0.66 () {56 vars in 1148 bytes} [Mon Jul 18 02:15:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:28.929618 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55a0379-b9bc-4dc7-a415-aba5ba18c8c8 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:28.934194 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:28.941490 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f89e887-15f9-47b8-9da9-1031b3160326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:28.942186 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f89e887-15f9-47b8-9da9-1031b3160326 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:28.942472 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f89e887-15f9-47b8-9da9-1031b3160326 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:28.942762 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f89e887-15f9-47b8-9da9-1031b3160326 None admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:28.943582 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f89e887-15f9-47b8-9da9-1031b3160326 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:28.943892 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f89e887-15f9-47b8-9da9-1031b3160326 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:28.945320 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f89e887-15f9-47b8-9da9-1031b3160326 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:28.967649 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 290/571] 10.222.0.66 () {56 vars in 1137 bytes} [Mon Jul 18 02:15:28 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:28.967649 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f89e887-15f9-47b8-9da9-1031b3160326 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:29.032416 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55a0379-b9bc-4dc7-a415-aba5ba18c8c8 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:29.062535 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f89e887-15f9-47b8-9da9-1031b3160326 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:29.066706 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a17a52-94ed-4c29-af24-e53ceecd39d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:29.067498 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a17a52-94ed-4c29-af24-e53ceecd39d7 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:29.067781 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a17a52-94ed-4c29-af24-e53ceecd39d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:29.068056 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a17a52-94ed-4c29-af24-e53ceecd39d7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:29.078811 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a17a52-94ed-4c29-af24-e53ceecd39d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 02:15:29.086621 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-75a17a52-94ed-4c29-af24-e53ceecd39d7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 02:15:29.086621 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:29.086621 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:29.086621 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:29.086621 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:29.086621 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:29.086621 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:29.086621 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:29.086621 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:29.086621 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:29.086621 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:29.086621 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:29.086621 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 02:15:29.086621 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 02:15:29.086621 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 02:15:29.086621 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 02:15:29.087583 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 02:15:29.087583 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 02:15:29.087583 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 02:15:29.087583 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 02:15:29.087583 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:29.087583 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:29.087583 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:29.087583 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:29.087583 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:29.087583 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:29.087583 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:29.087583 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:29.087583 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:29.087583 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:29.087583 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:29.087583 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:29.088449 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:29.088449 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:29.088449 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:29.088449 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:29.088449 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 02:15:29.088449 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 02:15:29.088449 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 02:15:29.088449 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 02:15:29.088449 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 02:15:29.088449 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 02:15:29.088449 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 02:15:29.088449 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:15:29.088449 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 282/572] 10.222.0.66 () {56 vars in 1102 bytes} [Mon Jul 18 02:15:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:29.089166 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852e3234-edca-44ae-a6e4-655260a5fec1 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:29.089455 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852e3234-edca-44ae-a6e4-655260a5fec1 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:29.089709 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852e3234-edca-44ae-a6e4-655260a5fec1 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:29.091138 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 283/573] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:15:29 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:15:29.094871 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852e3234-edca-44ae-a6e4-655260a5fec1 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:29.098963 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8de5a8f-d521-45d7-a6ae-19574491dfd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:29.099639 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8de5a8f-d521-45d7-a6ae-19574491dfd6 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:29.099924 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8de5a8f-d521-45d7-a6ae-19574491dfd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:29.100220 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8de5a8f-d521-45d7-a6ae-19574491dfd6 None admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:29.101083 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8de5a8f-d521-45d7-a6ae-19574491dfd6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:29.101392 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8de5a8f-d521-45d7-a6ae-19574491dfd6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:29.102849 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8de5a8f-d521-45d7-a6ae-19574491dfd6 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:29.120594 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 291/574] 10.222.0.66 () {56 vars in 1133 bytes} [Mon Jul 18 02:15:28 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:29.121384 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8de5a8f-d521-45d7-a6ae-19574491dfd6 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:29.150783 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:29.169640 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:29.198321 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852e3234-edca-44ae-a6e4-655260a5fec1 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:29.222475 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8de5a8f-d521-45d7-a6ae-19574491dfd6 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:29.231843 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038a955f-1206-43cb-a7da-30918ca1eb6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:29.232549 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038a955f-1206-43cb-a7da-30918ca1eb6c None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:29.232843 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038a955f-1206-43cb-a7da-30918ca1eb6c None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:29.233122 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038a955f-1206-43cb-a7da-30918ca1eb6c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:29.233949 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038a955f-1206-43cb-a7da-30918ca1eb6c None admin] RBAC: Adding query filter params (role.id=eea94daf3312421cb12aa8abfa73160e, scope.project.id=76f7c0ae36ae49b09dcae4c3e9a875f4, user.id=b4300622cbd347dd8f5227bb960e98d0) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:29.234296 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038a955f-1206-43cb-a7da-30918ca1eb6c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:29.235753 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038a955f-1206-43cb-a7da-30918ca1eb6c None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:29.241202 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 284/575] 10.222.0.66 () {56 vars in 1374 bytes} [Mon Jul 18 02:15:29 2022] GET /identity/v3/role_assignments?user.id=b4300622cbd347dd8f5227bb960e98d0&scope.project.id=76f7c0ae36ae49b09dcae4c3e9a875f4&role.id=eea94daf3312421cb12aa8abfa73160e => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:29.242146 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038a955f-1206-43cb-a7da-30918ca1eb6c None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:29.258945 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bb97a0-fd41-4825-b60c-b7f04a9e64fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:29.259793 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bb97a0-fd41-4825-b60c-b7f04a9e64fe None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:29.260075 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bb97a0-fd41-4825-b60c-b7f04a9e64fe None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:29.260703 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bb97a0-fd41-4825-b60c-b7f04a9e64fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:29.273525 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bb97a0-fd41-4825-b60c-b7f04a9e64fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:29.275348 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bb97a0-fd41-4825-b60c-b7f04a9e64fe None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:29.295106 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-b0bb97a0-fd41-4825-b60c-b7f04a9e64fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:15:29.295106 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:29.295106 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:29.295106 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:29.295106 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:29.295106 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:29.295106 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:29.295106 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:29.295106 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:29.295106 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:29.295106 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:29.295106 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:29.295106 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 02:15:29.295106 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 02:15:29.295106 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 02:15:29.296150 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 02:15:29.296150 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:29.296150 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:29.296150 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:29.296150 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:29.296150 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:29.296150 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:29.296150 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:29.296150 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:29.296150 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:29.296150 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:29.296150 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:29.296150 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:29.296150 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:29.296150 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:29.296150 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:29.296150 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:29.297043 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 02:15:29.297043 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 02:15:29.297043 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:15:29.297043 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:15:29.297043 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 292/576] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 18 02:15:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:29.298304 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39599136-7fa7-48c8-9b74-06eb768ad938 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:29.298423 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39599136-7fa7-48c8-9b74-06eb768ad938 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:29.298619 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39599136-7fa7-48c8-9b74-06eb768ad938 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:29.316139 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-39599136-7fa7-48c8-9b74-06eb768ad938 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:15:29.317376 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-39599136-7fa7-48c8-9b74-06eb768ad938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:15:29.363481 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038a955f-1206-43cb-a7da-30918ca1eb6c None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:29.383971 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:29.402262 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 293/577] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:15:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 02:15:29.402914 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39599136-7fa7-48c8-9b74-06eb768ad938 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:29.407955 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a53274-3921-4dcc-8ec3-f33ec2aa2266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:29.408634 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a53274-3921-4dcc-8ec3-f33ec2aa2266 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:29.408918 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a53274-3921-4dcc-8ec3-f33ec2aa2266 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:29.409192 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a53274-3921-4dcc-8ec3-f33ec2aa2266 None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:29.409943 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a53274-3921-4dcc-8ec3-f33ec2aa2266 None admin] RBAC: Adding query filter params (name=service) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:29.410280 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a53274-3921-4dcc-8ec3-f33ec2aa2266 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:29.411667 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a53274-3921-4dcc-8ec3-f33ec2aa2266 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:29.424165 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 285/578] 10.222.0.66 () {56 vars in 1103 bytes} [Mon Jul 18 02:15:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:29.424833 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a53274-3921-4dcc-8ec3-f33ec2aa2266 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:29.436290 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:29.515192 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39599136-7fa7-48c8-9b74-06eb768ad938 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:29.539558 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a53274-3921-4dcc-8ec3-f33ec2aa2266 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:29.553687 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d6c5a9-7649-4337-b8c3-aacd600895a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:29.554511 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d6c5a9-7649-4337-b8c3-aacd600895a4 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:29.554793 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d6c5a9-7649-4337-b8c3-aacd600895a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:29.555111 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d6c5a9-7649-4337-b8c3-aacd600895a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:29.566446 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d6c5a9-7649-4337-b8c3-aacd600895a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:29.567870 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d6c5a9-7649-4337-b8c3-aacd600895a4 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:29.577468 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7326d8be-69cf-4881-8d60-a1f49bdcc13a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:29.578169 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7326d8be-69cf-4881-8d60-a1f49bdcc13a None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:29.578456 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7326d8be-69cf-4881-8d60-a1f49bdcc13a None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:29.578734 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7326d8be-69cf-4881-8d60-a1f49bdcc13a None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:29.579461 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7326d8be-69cf-4881-8d60-a1f49bdcc13a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:29.579763 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7326d8be-69cf-4881-8d60-a1f49bdcc13a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:29.581133 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7326d8be-69cf-4881-8d60-a1f49bdcc13a None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:29.586947 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-72d6c5a9-7649-4337-b8c3-aacd600895a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:15:29.586947 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:29.586947 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:29.586947 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:29.586947 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:29.586947 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:29.586947 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:29.586947 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:29.586947 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:29.586947 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:29.586947 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:29.586947 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:29.586947 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 02:15:29.586947 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 02:15:29.586947 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 02:15:29.587809 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 02:15:29.587809 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:29.587809 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:29.587809 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:29.587809 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:29.587809 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:29.587809 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:29.587809 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:29.587809 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:29.587809 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:29.587809 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:29.587809 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:29.587809 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:29.587809 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:29.587809 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:29.587809 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:29.587809 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:29.588712 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 02:15:29.588712 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 02:15:29.588712 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:15:29.588712 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:15:29.588712 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 294/579] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 18 02:15:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:29.589013 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d6c5a9-7649-4337-b8c3-aacd600895a4 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:29.602530 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 286/580] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 18 02:15:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:29.602530 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7326d8be-69cf-4881-8d60-a1f49bdcc13a None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:29.649030 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:29.676933 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:29.699843 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d6c5a9-7649-4337-b8c3-aacd600895a4 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:29.714285 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7326d8be-69cf-4881-8d60-a1f49bdcc13a None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:29.736676 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c87a79-cd3f-4fcc-a41b-b7d48990ef80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:29.737470 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c87a79-cd3f-4fcc-a41b-b7d48990ef80 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:29.737759 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c87a79-cd3f-4fcc-a41b-b7d48990ef80 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:29.738034 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c87a79-cd3f-4fcc-a41b-b7d48990ef80 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:29.749080 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c87a79-cd3f-4fcc-a41b-b7d48990ef80 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 02:15:29.749625 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8847f49-71c6-40b9-9f14-88247724b87b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:29.750647 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8847f49-71c6-40b9-9f14-88247724b87b None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:29.751105 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8847f49-71c6-40b9-9f14-88247724b87b None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:29.751575 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8847f49-71c6-40b9-9f14-88247724b87b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:29.756805 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-31c87a79-cd3f-4fcc-a41b-b7d48990ef80 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 02:15:29.756805 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:29.756805 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:29.756805 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:29.756805 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:29.756805 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:29.756805 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:29.756805 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:29.756805 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:29.756805 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:29.756805 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:29.756805 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:29.756805 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 02:15:29.756805 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 02:15:29.756805 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 02:15:29.756805 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 02:15:29.757738 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 02:15:29.757738 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 02:15:29.757738 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 02:15:29.757738 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 02:15:29.757738 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:29.757738 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:29.757738 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:29.757738 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:29.757738 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:29.757738 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:29.757738 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:29.757738 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:29.757738 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:29.757738 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:29.757738 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:29.757738 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:29.758751 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:29.758751 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:29.758751 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:29.758751 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:29.758751 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 02:15:29.758751 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 02:15:29.758751 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 02:15:29.758751 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 02:15:29.758751 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 02:15:29.758751 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 02:15:29.758751 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 02:15:29.758751 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:15:29.758751 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 295/581] 10.222.0.66 () {56 vars in 1102 bytes} [Mon Jul 18 02:15:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:29.758751 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c87a79-cd3f-4fcc-a41b-b7d48990ef80 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:29.768425 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8847f49-71c6-40b9-9f14-88247724b87b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:29.770214 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8847f49-71c6-40b9-9f14-88247724b87b None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:29.787721 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-f8847f49-71c6-40b9-9f14-88247724b87b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:15:29.787721 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:29.787721 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:29.787721 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:29.787721 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:29.787721 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:29.787721 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:29.787721 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:29.787721 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:29.787721 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:29.787721 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:29.787721 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:29.787721 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 02:15:29.787721 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 02:15:29.787721 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 02:15:29.788833 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 02:15:29.788833 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:29.788833 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:29.788833 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:29.788833 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:29.788833 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:29.788833 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:29.788833 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:29.788833 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:29.788833 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:29.788833 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:29.788833 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:29.788833 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:29.788833 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:29.788833 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:29.788833 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:29.788833 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:29.789796 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 02:15:29.789796 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 02:15:29.789796 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:15:29.789796 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:15:29.789796 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 287/582] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 18 02:15:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:29.790108 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8847f49-71c6-40b9-9f14-88247724b87b None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:29.871620 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c87a79-cd3f-4fcc-a41b-b7d48990ef80 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:29.889532 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:29.899326 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8847f49-71c6-40b9-9f14-88247724b87b None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:29.907773 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e260cad6-5861-4658-aaaf-79c5bc1124e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:29.908497 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e260cad6-5861-4658-aaaf-79c5bc1124e1 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:29.908789 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e260cad6-5861-4658-aaaf-79c5bc1124e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:29.909066 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e260cad6-5861-4658-aaaf-79c5bc1124e1 None admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:29.909794 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e260cad6-5861-4658-aaaf-79c5bc1124e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:29.910126 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e260cad6-5861-4658-aaaf-79c5bc1124e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:29.911517 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e260cad6-5861-4658-aaaf-79c5bc1124e1 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:29.920012 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:29.925991 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 296/583] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 18 02:15:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:29.926992 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e260cad6-5861-4658-aaaf-79c5bc1124e1 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:29.942025 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de8dc29-524a-45ce-81f4-7723611112c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:29.942898 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de8dc29-524a-45ce-81f4-7723611112c4 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:29.943181 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de8dc29-524a-45ce-81f4-7723611112c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:29.943463 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de8dc29-524a-45ce-81f4-7723611112c4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:29.950700 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de8dc29-524a-45ce-81f4-7723611112c4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:29.952181 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de8dc29-524a-45ce-81f4-7723611112c4 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:29.965924 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-4de8dc29-524a-45ce-81f4-7723611112c4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 02:15:29.965924 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:29.965924 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:29.965924 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:29.965924 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:29.965924 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:29.965924 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:29.965924 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:29.965924 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:29.965924 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:29.965924 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:29.965924 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:29.965924 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 02:15:29.965924 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 02:15:29.965924 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 02:15:29.967157 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 02:15:29.967157 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:29.967157 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:29.967157 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:29.967157 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:29.967157 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:29.967157 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:29.967157 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:29.967157 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:29.967157 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:29.967157 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:29.967157 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:29.967157 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:29.967157 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:29.967157 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:29.967157 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:29.967157 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:29.968080 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 02:15:29.968080 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 02:15:29.968080 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 02:15:29.968080 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 02:15:29.968080 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 02:15:29.968080 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 02:15:29.968080 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 02:15:29.968080 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:15:29.968080 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 288/584] 10.222.0.66 () {56 vars in 1111 bytes} [Mon Jul 18 02:15:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:29.968585 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de8dc29-524a-45ce-81f4-7723611112c4 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:30.039907 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e260cad6-5861-4658-aaaf-79c5bc1124e1 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:30.075604 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de8dc29-524a-45ce-81f4-7723611112c4 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:30.077432 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7f258a-0878-4bda-8586-72ff6848e902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:30.078130 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7f258a-0878-4bda-8586-72ff6848e902 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:30.078416 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7f258a-0878-4bda-8586-72ff6848e902 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:30.078693 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7f258a-0878-4bda-8586-72ff6848e902 None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:30.079430 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7f258a-0878-4bda-8586-72ff6848e902 None admin] RBAC: Adding query filter params (name=service) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:30.079737 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7f258a-0878-4bda-8586-72ff6848e902 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:30.081109 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7f258a-0878-4bda-8586-72ff6848e902 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:30.089408 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 297/585] 10.222.0.66 () {56 vars in 1103 bytes} [Mon Jul 18 02:15:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:30.090176 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7f258a-0878-4bda-8586-72ff6848e902 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:30.117329 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3e4040-e663-4503-b883-abc2535d645a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:30.117992 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3e4040-e663-4503-b883-abc2535d645a None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:30.118309 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3e4040-e663-4503-b883-abc2535d645a None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:30.118590 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3e4040-e663-4503-b883-abc2535d645a None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:30.119311 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3e4040-e663-4503-b883-abc2535d645a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:30.119619 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3e4040-e663-4503-b883-abc2535d645a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:30.120994 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3e4040-e663-4503-b883-abc2535d645a None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:30.132981 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 289/586] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 18 02:15:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:30.133835 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3e4040-e663-4503-b883-abc2535d645a None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:30.149450 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:30.157711 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:30.199829 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7f258a-0878-4bda-8586-72ff6848e902 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:30.237413 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7b5df2-83d3-44f4-bc78-70e1fe73aea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:30.238247 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7b5df2-83d3-44f4-bc78-70e1fe73aea9 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:30.238533 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7b5df2-83d3-44f4-bc78-70e1fe73aea9 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:30.238806 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7b5df2-83d3-44f4-bc78-70e1fe73aea9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:30.242730 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3e4040-e663-4503-b883-abc2535d645a None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:30.246389 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7b5df2-83d3-44f4-bc78-70e1fe73aea9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:30.247887 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7b5df2-83d3-44f4-bc78-70e1fe73aea9 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:30.263447 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-9d7b5df2-83d3-44f4-bc78-70e1fe73aea9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 02:15:30.263447 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:30.263447 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:30.263447 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:30.263447 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:30.263447 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:30.263447 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:30.263447 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:30.263447 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:30.263447 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:30.263447 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:30.263447 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:30.263447 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 02:15:30.263447 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 02:15:30.263447 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 02:15:30.264369 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 02:15:30.264369 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:30.264369 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:30.264369 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:30.264369 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:30.264369 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:30.264369 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:30.264369 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:30.264369 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:30.264369 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:30.264369 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:30.264369 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:30.264369 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:30.264369 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:30.264369 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:30.264369 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:30.264369 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:30.265260 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 02:15:30.265260 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 02:15:30.265260 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 02:15:30.265260 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 02:15:30.265260 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 02:15:30.265260 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 02:15:30.265260 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 02:15:30.265260 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:15:30.265260 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 298/587] 10.222.0.66 () {56 vars in 1111 bytes} [Mon Jul 18 02:15:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:30.265758 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7b5df2-83d3-44f4-bc78-70e1fe73aea9 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:30.284368 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4a17a1-0c56-40da-984d-62f11d77478b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:30.285041 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4a17a1-0c56-40da-984d-62f11d77478b None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:30.285330 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4a17a1-0c56-40da-984d-62f11d77478b None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:30.285611 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4a17a1-0c56-40da-984d-62f11d77478b None admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:30.286435 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4a17a1-0c56-40da-984d-62f11d77478b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:30.286753 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4a17a1-0c56-40da-984d-62f11d77478b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:30.288197 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4a17a1-0c56-40da-984d-62f11d77478b None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:30.301934 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 290/588] 10.222.0.66 () {56 vars in 1148 bytes} [Mon Jul 18 02:15:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:30.302774 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4a17a1-0c56-40da-984d-62f11d77478b None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:30.379692 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7b5df2-83d3-44f4-bc78-70e1fe73aea9 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:30.385260 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:30.403511 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:30.419000 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83502dae-31f5-495b-befd-c6ec1e151897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:30.420342 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4a17a1-0c56-40da-984d-62f11d77478b None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:30.420518 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83502dae-31f5-495b-befd-c6ec1e151897 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:30.420819 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83502dae-31f5-495b-befd-c6ec1e151897 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:30.421096 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83502dae-31f5-495b-befd-c6ec1e151897 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:30.431511 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83502dae-31f5-495b-befd-c6ec1e151897 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:30.432957 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83502dae-31f5-495b-befd-c6ec1e151897 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:30.450966 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-83502dae-31f5-495b-befd-c6ec1e151897 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:15:30.450966 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:30.450966 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:30.450966 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:30.450966 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:30.450966 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:30.450966 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:30.450966 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:30.450966 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:30.450966 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:30.450966 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:30.450966 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:30.450966 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 02:15:30.450966 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 02:15:30.450966 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 02:15:30.452056 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 02:15:30.452056 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:30.452056 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:30.452056 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:30.452056 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:30.452056 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:30.452056 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:30.452056 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:30.452056 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:30.452056 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:30.452056 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:30.452056 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:30.452056 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:30.452056 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:30.452056 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:30.452056 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:30.452056 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:30.453014 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 02:15:30.453014 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 02:15:30.453014 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:15:30.453014 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:15:30.456626 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 299/589] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 18 02:15:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:30.457330 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83502dae-31f5-495b-befd-c6ec1e151897 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:30.459250 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d63fac-ef74-4592-8a6a-be734c384c45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:30.460053 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d63fac-ef74-4592-8a6a-be734c384c45 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:30.460335 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d63fac-ef74-4592-8a6a-be734c384c45 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:30.460610 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d63fac-ef74-4592-8a6a-be734c384c45 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:30.470446 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d63fac-ef74-4592-8a6a-be734c384c45 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:30.471896 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d63fac-ef74-4592-8a6a-be734c384c45 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:30.489671 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-d3d63fac-ef74-4592-8a6a-be734c384c45 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 18 02:15:30.489671 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:30.489671 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:30.489671 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:30.489671 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:30.489671 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:30.489671 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:30.489671 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:30.489671 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:30.489671 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:30.489671 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:30.489671 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:30.489671 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 02:15:30.489671 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 02:15:30.489671 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 02:15:30.489671 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 02:15:30.490848 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:30.490848 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:30.490848 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 02:15:30.490848 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 02:15:30.490848 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 02:15:30.490848 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 02:15:30.490848 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:30.490848 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:30.490848 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:30.490848 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:30.490848 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:30.490848 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:30.490848 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:30.490848 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:30.490848 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:30.490848 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:30.490848 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:30.490848 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:30.491821 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:30.491821 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:30.491821 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 02:15:30.491821 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 02:15:30.491821 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 02:15:30.491821 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 02:15:30.491821 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 02:15:30.491821 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 02:15:30.491821 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 18 02:15:30.491821 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:15:30.491821 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 291/590] 10.222.0.66 () {56 vars in 1099 bytes} [Mon Jul 18 02:15:30 2022] GET /identity/v3/users/cinder => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:30.492442 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d63fac-ef74-4592-8a6a-be734c384c45 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:30.575966 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83502dae-31f5-495b-befd-c6ec1e151897 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:30.602235 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d63fac-ef74-4592-8a6a-be734c384c45 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:30.619832 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34476c2b-c1f4-463b-aeca-8a33e898746d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:30.620497 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34476c2b-c1f4-463b-aeca-8a33e898746d None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:30.620774 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34476c2b-c1f4-463b-aeca-8a33e898746d None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:30.621045 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34476c2b-c1f4-463b-aeca-8a33e898746d None admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:30.621825 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34476c2b-c1f4-463b-aeca-8a33e898746d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:30.622150 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34476c2b-c1f4-463b-aeca-8a33e898746d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:30.623698 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34476c2b-c1f4-463b-aeca-8a33e898746d None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:30.635138 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 300/591] 10.222.0.66 () {56 vars in 1148 bytes} [Mon Jul 18 02:15:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:30.635917 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34476c2b-c1f4-463b-aeca-8a33e898746d None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:30.641472 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:30.648404 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a738efe-ca34-4a18-8e93-13785b362a5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:30.652004 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a738efe-ca34-4a18-8e93-13785b362a5e None admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:30.652004 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a738efe-ca34-4a18-8e93-13785b362a5e None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:30.652004 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a738efe-ca34-4a18-8e93-13785b362a5e None admin] PATH_INFO: `/v3/projects/76f7c0ae36ae49b09dcae4c3e9a875f4/users/e1391b81e7234bc18e22b9e84f231256/roles/eea94daf3312421cb12aa8abfa73160e` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:30.667451 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:30.668394 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a738efe-ca34-4a18-8e93-13785b362a5e None admin] RBAC: Authorizing `identity:create_grant(project_id=76f7c0ae36ae49b09dcae4c3e9a875f4, user_id=e1391b81e7234bc18e22b9e84f231256, role_id=eea94daf3312421cb12aa8abfa73160e)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:30.671494 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a738efe-ca34-4a18-8e93-13785b362a5e None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:30.739878 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34476c2b-c1f4-463b-aeca-8a33e898746d None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:30.758806 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 292/592] 10.222.0.66 () {58 vars in 1439 bytes} [Mon Jul 18 02:15:30 2022] PUT /identity/v3/projects/76f7c0ae36ae49b09dcae4c3e9a875f4/users/e1391b81e7234bc18e22b9e84f231256/roles/eea94daf3312421cb12aa8abfa73160e => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:15:30.759645 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a738efe-ca34-4a18-8e93-13785b362a5e None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:30.789034 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee2e8a2-d6b6-4ed8-9234-81e19017ef2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:30.790465 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee2e8a2-d6b6-4ed8-9234-81e19017ef2f None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:30.790899 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee2e8a2-d6b6-4ed8-9234-81e19017ef2f None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:30.791888 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee2e8a2-d6b6-4ed8-9234-81e19017ef2f None admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:30.793093 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee2e8a2-d6b6-4ed8-9234-81e19017ef2f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:30.793834 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee2e8a2-d6b6-4ed8-9234-81e19017ef2f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:30.796868 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee2e8a2-d6b6-4ed8-9234-81e19017ef2f None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:30.815306 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 301/593] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 18 02:15:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:30.816603 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee2e8a2-d6b6-4ed8-9234-81e19017ef2f None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:30.895849 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a738efe-ca34-4a18-8e93-13785b362a5e None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:30.924608 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:30.936364 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee2e8a2-d6b6-4ed8-9234-81e19017ef2f None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:30.945808 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2c6da9-346b-4372-89a6-543c527400ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:30.946644 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2c6da9-346b-4372-89a6-543c527400ac None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:30.947003 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2c6da9-346b-4372-89a6-543c527400ac None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:30.947535 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2c6da9-346b-4372-89a6-543c527400ac None admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:30.948135 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2c6da9-346b-4372-89a6-543c527400ac None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:30.948447 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2c6da9-346b-4372-89a6-543c527400ac None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:30.949907 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2c6da9-346b-4372-89a6-543c527400ac None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:30.976435 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 293/594] 10.222.0.66 () {56 vars in 1137 bytes} [Mon Jul 18 02:15:30 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:30.977597 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2c6da9-346b-4372-89a6-543c527400ac None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:30.988792 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71c5012-f269-4e58-83ec-33cf18320e58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:30.989702 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71c5012-f269-4e58-83ec-33cf18320e58 None admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:30.990037 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71c5012-f269-4e58-83ec-33cf18320e58 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:30.990326 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71c5012-f269-4e58-83ec-33cf18320e58 None admin] PATH_INFO: `/v3/projects/76f7c0ae36ae49b09dcae4c3e9a875f4/users/80b138831bc54c7b986d50092d62d4af/roles/eea94daf3312421cb12aa8abfa73160e` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:31.010846 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71c5012-f269-4e58-83ec-33cf18320e58 None admin] RBAC: Authorizing `identity:create_grant(project_id=76f7c0ae36ae49b09dcae4c3e9a875f4, user_id=80b138831bc54c7b986d50092d62d4af, role_id=eea94daf3312421cb12aa8abfa73160e)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:31.013003 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71c5012-f269-4e58-83ec-33cf18320e58 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:31.025636 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:31.084630 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2c6da9-346b-4372-89a6-543c527400ac None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:31.097079 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 302/595] 10.222.0.66 () {58 vars in 1439 bytes} [Mon Jul 18 02:15:30 2022] PUT /identity/v3/projects/76f7c0ae36ae49b09dcae4c3e9a875f4/users/80b138831bc54c7b986d50092d62d4af/roles/eea94daf3312421cb12aa8abfa73160e => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:15:31.098664 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71c5012-f269-4e58-83ec-33cf18320e58 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:31.123556 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58506535-18c8-459f-970f-bbe40d9032e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:31.124440 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58506535-18c8-459f-970f-bbe40d9032e7 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:31.124734 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58506535-18c8-459f-970f-bbe40d9032e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:31.125398 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58506535-18c8-459f-970f-bbe40d9032e7 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:31.134131 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58506535-18c8-459f-970f-bbe40d9032e7 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:31.135786 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58506535-18c8-459f-970f-bbe40d9032e7 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:31.153715 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-58506535-18c8-459f-970f-bbe40d9032e7 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 18 02:15:31.153715 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:31.153715 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:31.153715 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:31.153715 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:31.153715 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:31.153715 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:31.153715 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:31.153715 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:31.153715 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:31.153715 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:31.153715 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:31.153715 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 02:15:31.153715 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 02:15:31.153715 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 02:15:31.153715 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 02:15:31.154886 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:31.154886 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:31.154886 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 02:15:31.154886 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 02:15:31.154886 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 02:15:31.154886 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 02:15:31.154886 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:31.154886 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:31.154886 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:31.154886 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:31.154886 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:31.154886 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:31.154886 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:31.154886 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:31.154886 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:31.154886 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:31.154886 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:31.154886 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:31.155834 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:31.155834 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:31.155834 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 02:15:31.155834 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 02:15:31.155834 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 02:15:31.155834 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 02:15:31.155834 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 02:15:31.155834 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 02:15:31.155834 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 18 02:15:31.155834 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:15:31.159342 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 294/596] 10.222.0.66 () {56 vars in 1102 bytes} [Mon Jul 18 02:15:30 2022] GET /identity/v3/users/neutron => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:31.164412 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58506535-18c8-459f-970f-bbe40d9032e7 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:31.234956 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71c5012-f269-4e58-83ec-33cf18320e58 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:31.274951 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869c37f7-3e5f-469e-809c-a6342dfcc752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:31.276173 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869c37f7-3e5f-469e-809c-a6342dfcc752 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:31.276663 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869c37f7-3e5f-469e-809c-a6342dfcc752 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:31.277079 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869c37f7-3e5f-469e-809c-a6342dfcc752 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:31.289780 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869c37f7-3e5f-469e-809c-a6342dfcc752 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:31.291500 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869c37f7-3e5f-469e-809c-a6342dfcc752 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:31.310949 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:31.315054 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-869c37f7-3e5f-469e-809c-a6342dfcc752 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:15:31.315054 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:31.315054 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:31.315054 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:31.315054 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:31.315054 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:31.315054 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:31.315054 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:31.315054 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:31.315054 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:31.315054 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:31.315054 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:31.315054 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 02:15:31.315054 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 02:15:31.315054 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 02:15:31.315948 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 02:15:31.315948 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:31.315948 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:31.315948 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:31.315948 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:31.315948 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:31.315948 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:31.315948 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:31.315948 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:31.315948 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:31.315948 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:31.315948 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:31.315948 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:31.315948 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:31.315948 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:31.315948 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:31.315948 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:31.316846 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 02:15:31.316846 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 02:15:31.316846 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:15:31.316846 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:15:31.317481 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 303/597] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 18 02:15:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:31.322826 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869c37f7-3e5f-469e-809c-a6342dfcc752 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:31.357414 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:31.368362 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58506535-18c8-459f-970f-bbe40d9032e7 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:31.424411 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725f4838-e657-48f9-8c0c-bbb8078aaafc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:31.425209 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725f4838-e657-48f9-8c0c-bbb8078aaafc None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:31.425610 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725f4838-e657-48f9-8c0c-bbb8078aaafc None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:31.425886 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725f4838-e657-48f9-8c0c-bbb8078aaafc None admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:31.426768 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725f4838-e657-48f9-8c0c-bbb8078aaafc None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:31.427083 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725f4838-e657-48f9-8c0c-bbb8078aaafc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:31.428915 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725f4838-e657-48f9-8c0c-bbb8078aaafc None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:31.457935 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 295/598] 10.222.0.66 () {56 vars in 1139 bytes} [Mon Jul 18 02:15:31 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:31.459715 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a8733b-5998-44cf-ad44-2552b8a00fdc None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:31.460146 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a8733b-5998-44cf-ad44-2552b8a00fdc None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:31.460568 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a8733b-5998-44cf-ad44-2552b8a00fdc None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:31.466369 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 296/599] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:15:31 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:15:31.467185 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a8733b-5998-44cf-ad44-2552b8a00fdc None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:31.474800 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869c37f7-3e5f-469e-809c-a6342dfcc752 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:31.511625 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a2d64e-d45e-43a6-9c43-5fa0e62e3001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:31.512446 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a2d64e-d45e-43a6-9c43-5fa0e62e3001 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:31.512837 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a2d64e-d45e-43a6-9c43-5fa0e62e3001 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:31.513715 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a2d64e-d45e-43a6-9c43-5fa0e62e3001 None admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:31.514469 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a2d64e-d45e-43a6-9c43-5fa0e62e3001 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:31.514878 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a2d64e-d45e-43a6-9c43-5fa0e62e3001 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:31.516365 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a2d64e-d45e-43a6-9c43-5fa0e62e3001 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:31.529732 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 304/600] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 18 02:15:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:31.531066 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454f0116-ea40-40a9-8b09-696b475899a4 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:31.531351 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454f0116-ea40-40a9-8b09-696b475899a4 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:31.531627 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454f0116-ea40-40a9-8b09-696b475899a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:31.549642 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-454f0116-ea40-40a9-8b09-696b475899a4 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:15:31.550966 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-454f0116-ea40-40a9-8b09-696b475899a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:15:31.579117 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a8733b-5998-44cf-ad44-2552b8a00fdc None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:31.612989 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:31.622841 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4cf7e0-c048-48b5-a9d4-87a62da33854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:31.626960 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4cf7e0-c048-48b5-a9d4-87a62da33854 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:31.626960 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4cf7e0-c048-48b5-a9d4-87a62da33854 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:31.626960 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4cf7e0-c048-48b5-a9d4-87a62da33854 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:31.638374 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 305/601] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:15:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 02:15:31.638374 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454f0116-ea40-40a9-8b09-696b475899a4 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:31.641743 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4cf7e0-c048-48b5-a9d4-87a62da33854 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:31.643440 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4cf7e0-c048-48b5-a9d4-87a62da33854 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:31.678707 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-7a4cf7e0-c048-48b5-a9d4-87a62da33854 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:15:31.678707 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:31.678707 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:31.678707 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:31.678707 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:31.678707 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:31.678707 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:31.678707 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:31.678707 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:31.678707 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:31.678707 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:31.678707 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:31.678707 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 02:15:31.678707 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 02:15:31.678707 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 02:15:31.680253 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 02:15:31.680253 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:31.680253 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:31.680253 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:31.680253 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:31.680253 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:31.680253 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:31.680253 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:31.680253 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:31.680253 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:31.680253 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:31.680253 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:31.680253 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:31.680253 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:31.680253 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:31.680253 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:31.680253 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:31.681331 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 02:15:31.681331 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 02:15:31.681331 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:15:31.681331 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:15:31.683281 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 297/602] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 18 02:15:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:31.688790 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4cf7e0-c048-48b5-a9d4-87a62da33854 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:31.704386 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:31.742059 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454f0116-ea40-40a9-8b09-696b475899a4 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:31.780766 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b27cad7-0322-4b43-b268-595dfe01ed1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:31.781483 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b27cad7-0322-4b43-b268-595dfe01ed1e None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:31.781762 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b27cad7-0322-4b43-b268-595dfe01ed1e None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:31.782032 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b27cad7-0322-4b43-b268-595dfe01ed1e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:31.789310 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b27cad7-0322-4b43-b268-595dfe01ed1e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:31.790894 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b27cad7-0322-4b43-b268-595dfe01ed1e None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:31.803549 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4cf7e0-c048-48b5-a9d4-87a62da33854 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:31.805735 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-9b27cad7-0322-4b43-b268-595dfe01ed1e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 02:15:31.805735 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:31.805735 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:31.805735 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:31.805735 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:31.805735 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:31.805735 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:31.805735 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:31.805735 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:31.805735 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:31.805735 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:31.805735 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:31.805735 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 02:15:31.805735 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 02:15:31.805735 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 02:15:31.806832 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 02:15:31.806832 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:31.806832 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:31.806832 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:31.806832 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:31.806832 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:31.806832 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:31.806832 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:31.806832 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:31.806832 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:31.806832 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:31.806832 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:31.806832 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:31.806832 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:31.806832 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:31.806832 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:31.806832 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:31.807811 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 02:15:31.807811 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 02:15:31.807811 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 02:15:31.807811 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 02:15:31.807811 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 02:15:31.807811 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 02:15:31.807811 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 02:15:31.807811 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:15:31.807811 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 306/603] 10.222.0.66 () {56 vars in 1111 bytes} [Mon Jul 18 02:15:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:31.808311 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b27cad7-0322-4b43-b268-595dfe01ed1e None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:31.850135 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee140f4-cc50-4b15-956d-1ec81394d6bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:31.851057 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee140f4-cc50-4b15-956d-1ec81394d6bf None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:31.851509 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee140f4-cc50-4b15-956d-1ec81394d6bf None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:31.851962 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee140f4-cc50-4b15-956d-1ec81394d6bf None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:31.852833 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee140f4-cc50-4b15-956d-1ec81394d6bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:31.853290 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee140f4-cc50-4b15-956d-1ec81394d6bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:31.855372 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee140f4-cc50-4b15-956d-1ec81394d6bf None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:31.861378 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:31.876420 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 298/604] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 18 02:15:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 02:15:31.876420 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee140f4-cc50-4b15-956d-1ec81394d6bf None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:31.930137 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b27cad7-0322-4b43-b268-595dfe01ed1e None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:31.959032 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:31.968349 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4321b81c-26fd-48b5-8dd0-7e7b0bef0902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:31.969233 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4321b81c-26fd-48b5-8dd0-7e7b0bef0902 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:31.973062 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4321b81c-26fd-48b5-8dd0-7e7b0bef0902 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:31.973062 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4321b81c-26fd-48b5-8dd0-7e7b0bef0902 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:31.982323 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4321b81c-26fd-48b5-8dd0-7e7b0bef0902 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 02:15:31.990537 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-4321b81c-26fd-48b5-8dd0-7e7b0bef0902 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 02:15:31.990537 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:31.990537 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:31.990537 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:31.990537 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:31.990537 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:31.990537 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:31.990537 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:31.990537 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:31.990537 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:31.990537 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:31.990537 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:31.990537 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 02:15:31.990537 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 02:15:31.990537 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 02:15:31.990537 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 02:15:31.991570 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 02:15:31.991570 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 02:15:31.991570 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 02:15:31.991570 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 02:15:31.991570 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:31.991570 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:31.991570 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:31.991570 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:31.991570 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:31.991570 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:31.991570 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:31.991570 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:31.991570 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:31.991570 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:31.991570 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:31.991570 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:31.992434 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:31.992434 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:31.992434 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:31.992434 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:31.992434 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 02:15:31.992434 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 02:15:31.992434 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 02:15:31.992434 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 02:15:31.992434 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 02:15:31.992434 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 02:15:31.992434 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 02:15:31.992434 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:15:31.997172 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 307/605] 10.222.0.66 () {56 vars in 1102 bytes} [Mon Jul 18 02:15:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:31.999450 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4321b81c-26fd-48b5-8dd0-7e7b0bef0902 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:32.003403 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee140f4-cc50-4b15-956d-1ec81394d6bf None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:32.040066 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3c2da8-6f22-45b4-892b-3e382de657c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:32.040647 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3c2da8-6f22-45b4-892b-3e382de657c4 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:32.040933 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3c2da8-6f22-45b4-892b-3e382de657c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:32.041210 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3c2da8-6f22-45b4-892b-3e382de657c4 None admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:32.041994 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3c2da8-6f22-45b4-892b-3e382de657c4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:32.042353 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3c2da8-6f22-45b4-892b-3e382de657c4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:32.043792 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3c2da8-6f22-45b4-892b-3e382de657c4 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:32.059388 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 299/606] 10.222.0.66 () {56 vars in 1148 bytes} [Mon Jul 18 02:15:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:32.060882 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3c2da8-6f22-45b4-892b-3e382de657c4 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:32.129344 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4321b81c-26fd-48b5-8dd0-7e7b0bef0902 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:32.132817 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:32.169296 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294ba321-7219-40ef-b29f-d34ad63ec874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:32.170725 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294ba321-7219-40ef-b29f-d34ad63ec874 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:32.171467 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294ba321-7219-40ef-b29f-d34ad63ec874 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:32.172591 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294ba321-7219-40ef-b29f-d34ad63ec874 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:32.181691 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294ba321-7219-40ef-b29f-d34ad63ec874 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:32.183640 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294ba321-7219-40ef-b29f-d34ad63ec874 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:32.190208 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3c2da8-6f22-45b4-892b-3e382de657c4 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:32.198928 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-294ba321-7219-40ef-b29f-d34ad63ec874 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 02:15:32.198928 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:32.198928 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:32.198928 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:32.198928 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:32.198928 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:32.198928 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:32.198928 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:32.198928 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:32.198928 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:32.198928 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:32.198928 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:32.198928 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 02:15:32.198928 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 02:15:32.198928 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 02:15:32.199852 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 02:15:32.199852 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:32.199852 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:32.199852 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:32.199852 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:32.199852 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:32.199852 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:32.199852 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:32.199852 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:32.199852 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:32.199852 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:32.199852 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:32.199852 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:32.199852 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:32.199852 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:32.199852 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:32.199852 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:32.201372 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 02:15:32.201372 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 02:15:32.201372 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 02:15:32.201372 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 02:15:32.201372 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 02:15:32.201372 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 02:15:32.201372 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 02:15:32.201372 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:15:32.205536 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 308/607] 10.222.0.66 () {56 vars in 1111 bytes} [Mon Jul 18 02:15:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:32.206436 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294ba321-7219-40ef-b29f-d34ad63ec874 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:32.225808 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:32.236950 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f39c34-9667-4d7f-bb53-f3ddecf6511a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:32.238106 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f39c34-9667-4d7f-bb53-f3ddecf6511a None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:32.242126 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f39c34-9667-4d7f-bb53-f3ddecf6511a None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:32.242126 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f39c34-9667-4d7f-bb53-f3ddecf6511a None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:32.242126 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f39c34-9667-4d7f-bb53-f3ddecf6511a None admin] RBAC: Adding query filter params (name=service) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:32.242126 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f39c34-9667-4d7f-bb53-f3ddecf6511a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:32.242126 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f39c34-9667-4d7f-bb53-f3ddecf6511a None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:32.253703 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 300/608] 10.222.0.66 () {56 vars in 1103 bytes} [Mon Jul 18 02:15:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:32.255033 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f39c34-9667-4d7f-bb53-f3ddecf6511a None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:32.318910 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294ba321-7219-40ef-b29f-d34ad63ec874 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:32.359842 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2eaea46-1447-4dbe-924c-e396bf3c228a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:32.361129 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eaea46-1447-4dbe-924c-e396bf3c228a None admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:32.361490 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eaea46-1447-4dbe-924c-e396bf3c228a None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:32.361898 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eaea46-1447-4dbe-924c-e396bf3c228a None admin] PATH_INFO: `/v3/projects/76f7c0ae36ae49b09dcae4c3e9a875f4/users/cd49388facdf446885e55b45ef91f01e/roles/eea94daf3312421cb12aa8abfa73160e` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:32.383341 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2eaea46-1447-4dbe-924c-e396bf3c228a None admin] RBAC: Authorizing `identity:create_grant(project_id=76f7c0ae36ae49b09dcae4c3e9a875f4, user_id=cd49388facdf446885e55b45ef91f01e, role_id=eea94daf3312421cb12aa8abfa73160e)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:32.385163 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2eaea46-1447-4dbe-924c-e396bf3c228a None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:32.400949 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f39c34-9667-4d7f-bb53-f3ddecf6511a None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:32.440249 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfd4ea5-544a-4637-8289-636eef2cf86c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:32.440915 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfd4ea5-544a-4637-8289-636eef2cf86c None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:32.441201 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfd4ea5-544a-4637-8289-636eef2cf86c None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:32.441480 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfd4ea5-544a-4637-8289-636eef2cf86c None admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:32.442361 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfd4ea5-544a-4637-8289-636eef2cf86c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:32.442675 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfd4ea5-544a-4637-8289-636eef2cf86c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:32.444192 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfd4ea5-544a-4637-8289-636eef2cf86c None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:32.457279 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 301/609] 10.222.0.66 () {56 vars in 1148 bytes} [Mon Jul 18 02:15:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:32.458238 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfd4ea5-544a-4637-8289-636eef2cf86c None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:32.498111 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 309/610] 10.222.0.66 () {58 vars in 1439 bytes} [Mon Jul 18 02:15:32 2022] PUT /identity/v3/projects/76f7c0ae36ae49b09dcae4c3e9a875f4/users/cd49388facdf446885e55b45ef91f01e/roles/eea94daf3312421cb12aa8abfa73160e => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:15:32.499297 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2eaea46-1447-4dbe-924c-e396bf3c228a None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:32.521717 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:32.556062 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:32.645095 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2eaea46-1447-4dbe-924c-e396bf3c228a None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:32.690761 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5288992d-8970-42ac-a7df-2d6c676ed497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:32.691716 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5288992d-8970-42ac-a7df-2d6c676ed497 None admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:32.692004 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5288992d-8970-42ac-a7df-2d6c676ed497 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:32.692281 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5288992d-8970-42ac-a7df-2d6c676ed497 None admin] PATH_INFO: `/v3/projects/76f7c0ae36ae49b09dcae4c3e9a875f4/users/eaf19a259c2741cea375a38763e33cdc/roles/eea94daf3312421cb12aa8abfa73160e` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:32.711514 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5288992d-8970-42ac-a7df-2d6c676ed497 None admin] RBAC: Authorizing `identity:create_grant(project_id=76f7c0ae36ae49b09dcae4c3e9a875f4, user_id=eaf19a259c2741cea375a38763e33cdc, role_id=eea94daf3312421cb12aa8abfa73160e)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:32.713060 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5288992d-8970-42ac-a7df-2d6c676ed497 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:32.725547 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfd4ea5-544a-4637-8289-636eef2cf86c None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:32.765599 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7f0edf-4b91-472f-9c6f-97684c2d8a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:32.766492 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7f0edf-4b91-472f-9c6f-97684c2d8a94 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:32.766784 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7f0edf-4b91-472f-9c6f-97684c2d8a94 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:32.767061 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7f0edf-4b91-472f-9c6f-97684c2d8a94 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:32.779062 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7f0edf-4b91-472f-9c6f-97684c2d8a94 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:32.780932 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7f0edf-4b91-472f-9c6f-97684c2d8a94 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:32.801437 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-9e7f0edf-4b91-472f-9c6f-97684c2d8a94 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:15:32.801437 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:32.801437 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:32.801437 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:32.801437 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:32.801437 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:32.801437 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:32.801437 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:32.801437 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:32.801437 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:32.801437 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:32.801437 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:32.801437 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 02:15:32.801437 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 02:15:32.801437 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 02:15:32.802503 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 02:15:32.802503 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:32.802503 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:32.802503 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:32.802503 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:32.802503 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:32.802503 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:32.802503 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:32.802503 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:32.802503 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:32.802503 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:32.802503 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:32.802503 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:32.802503 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:32.802503 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:32.802503 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:32.802503 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:32.803564 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 02:15:32.803564 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 02:15:32.803564 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:15:32.803564 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:15:32.803564 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 302/611] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 18 02:15:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 345 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:32.812905 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 310/612] 10.222.0.66 () {58 vars in 1439 bytes} [Mon Jul 18 02:15:32 2022] PUT /identity/v3/projects/76f7c0ae36ae49b09dcae4c3e9a875f4/users/eaf19a259c2741cea375a38763e33cdc/roles/eea94daf3312421cb12aa8abfa73160e => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:15:32.813836 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7f0edf-4b91-472f-9c6f-97684c2d8a94 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:32.934637 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581f6702-0eaa-4192-8292-988ff06b5006 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:32.935040 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581f6702-0eaa-4192-8292-988ff06b5006 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:32.935278 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581f6702-0eaa-4192-8292-988ff06b5006 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:32.937026 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 311/613] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:15:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 02:15:32.942638 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39283655-73b4-48b3-95c0-9a5e9ba7377b None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:32.942926 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39283655-73b4-48b3-95c0-9a5e9ba7377b None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:32.943206 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39283655-73b4-48b3-95c0-9a5e9ba7377b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:32.949133 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7f0edf-4b91-472f-9c6f-97684c2d8a94 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:32.951992 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:32.983559 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-39283655-73b4-48b3-95c0-9a5e9ba7377b None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:15:32.986325 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-39283655-73b4-48b3-95c0-9a5e9ba7377b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:15:33.013293 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c94861-179d-41e3-b75d-728988c21d3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:33.014427 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c94861-179d-41e3-b75d-728988c21d3b None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:33.014898 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c94861-179d-41e3-b75d-728988c21d3b None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:33.015354 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c94861-179d-41e3-b75d-728988c21d3b None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:33.016284 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c94861-179d-41e3-b75d-728988c21d3b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:33.016785 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c94861-179d-41e3-b75d-728988c21d3b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:33.018487 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c94861-179d-41e3-b75d-728988c21d3b None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:33.033108 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 303/614] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 18 02:15:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:33.038276 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c94861-179d-41e3-b75d-728988c21d3b None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:33.094573 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:33.105990 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 312/615] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:15:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 02:15:33.134335 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39283655-73b4-48b3-95c0-9a5e9ba7377b None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:33.178386 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c94861-179d-41e3-b75d-728988c21d3b None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:33.217392 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc9a3a0-fcab-420b-a875-3c8fe85a0f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:33.218133 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc9a3a0-fcab-420b-a875-3c8fe85a0f8a None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:33.218428 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc9a3a0-fcab-420b-a875-3c8fe85a0f8a None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:33.218711 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc9a3a0-fcab-420b-a875-3c8fe85a0f8a None admin] PATH_INFO: `/v3/users/swift` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:33.227079 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc9a3a0-fcab-420b-a875-3c8fe85a0f8a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:33.228674 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc9a3a0-fcab-420b-a875-3c8fe85a0f8a None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:33.244843 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-adc9a3a0-fcab-420b-a875-3c8fe85a0f8a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 18 02:15:33.244843 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:33.244843 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:33.244843 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:33.244843 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:33.244843 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:33.244843 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:33.244843 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:33.244843 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:33.244843 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:33.244843 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:33.244843 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:33.244843 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 02:15:33.244843 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 02:15:33.244843 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 02:15:33.244843 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 02:15:33.245915 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:33.245915 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:33.245915 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 02:15:33.245915 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 02:15:33.245915 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 02:15:33.245915 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 02:15:33.245915 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:33.245915 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:33.245915 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:33.245915 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:33.245915 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:33.245915 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:33.245915 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:33.245915 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:33.245915 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:33.245915 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:33.245915 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:33.245915 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:33.246893 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:33.246893 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:33.246893 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 02:15:33.246893 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 02:15:33.246893 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 02:15:33.246893 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 02:15:33.246893 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 02:15:33.246893 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 02:15:33.246893 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 18 02:15:33.246893 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:15:33.246893 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 304/616] 10.222.0.66 () {56 vars in 1096 bytes} [Mon Jul 18 02:15:33 2022] GET /identity/v3/users/swift => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:33.252201 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc9a3a0-fcab-420b-a875-3c8fe85a0f8a None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:33.260158 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39283655-73b4-48b3-95c0-9a5e9ba7377b None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:33.298562 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:33.311737 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d612b59-8ce6-4c91-8380-3b3edefc44c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:33.312552 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d612b59-8ce6-4c91-8380-3b3edefc44c7 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:33.312838 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d612b59-8ce6-4c91-8380-3b3edefc44c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:33.313118 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d612b59-8ce6-4c91-8380-3b3edefc44c7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:33.325732 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d612b59-8ce6-4c91-8380-3b3edefc44c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 02:15:33.334162 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-8d612b59-8ce6-4c91-8380-3b3edefc44c7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 02:15:33.334162 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:33.334162 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:33.334162 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:33.334162 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:33.334162 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:33.334162 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:33.334162 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:33.334162 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:33.334162 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:33.334162 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:33.334162 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:33.334162 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 02:15:33.334162 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 02:15:33.334162 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 02:15:33.334162 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 02:15:33.335286 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 02:15:33.335286 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 02:15:33.335286 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 02:15:33.335286 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 02:15:33.335286 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:33.335286 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:33.335286 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:33.335286 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:33.335286 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:33.335286 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:33.335286 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:33.335286 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:33.335286 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:33.335286 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:33.335286 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:33.335286 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:33.336736 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:33.336736 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:33.336736 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:33.336736 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:33.336736 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 02:15:33.336736 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 02:15:33.336736 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 02:15:33.336736 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 02:15:33.336736 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 02:15:33.336736 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 02:15:33.336736 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 02:15:33.336736 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:15:33.341433 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 313/617] 10.222.0.66 () {56 vars in 1102 bytes} [Mon Jul 18 02:15:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:33.342944 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d612b59-8ce6-4c91-8380-3b3edefc44c7 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:33.416636 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:33.417531 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc9a3a0-fcab-420b-a875-3c8fe85a0f8a None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:33.455792 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9167274d-2104-4341-aa38-778b3729b092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:33.456466 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9167274d-2104-4341-aa38-778b3729b092 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:33.456751 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9167274d-2104-4341-aa38-778b3729b092 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:33.457129 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9167274d-2104-4341-aa38-778b3729b092 None admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:33.458076 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9167274d-2104-4341-aa38-778b3729b092 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:33.458420 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9167274d-2104-4341-aa38-778b3729b092 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:33.459973 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9167274d-2104-4341-aa38-778b3729b092 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:33.476047 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d612b59-8ce6-4c91-8380-3b3edefc44c7 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:33.485378 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 305/618] 10.222.0.66 () {56 vars in 1135 bytes} [Mon Jul 18 02:15:33 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 02:15:33.487085 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316c0b7f-6f22-425b-81f7-f3243089278f None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:33.487374 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316c0b7f-6f22-425b-81f7-f3243089278f None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:33.487636 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316c0b7f-6f22-425b-81f7-f3243089278f None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:33.489296 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 306/619] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:15:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:15:33.489992 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316c0b7f-6f22-425b-81f7-f3243089278f None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:33.517369 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d026281a-9bcb-4a41-933f-580d1ce6a1d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:33.518787 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d026281a-9bcb-4a41-933f-580d1ce6a1d1 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:33.519171 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d026281a-9bcb-4a41-933f-580d1ce6a1d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:33.519452 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d026281a-9bcb-4a41-933f-580d1ce6a1d1 None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:33.520617 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d026281a-9bcb-4a41-933f-580d1ce6a1d1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:33.520930 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d026281a-9bcb-4a41-933f-580d1ce6a1d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:33.523060 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d026281a-9bcb-4a41-933f-580d1ce6a1d1 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:33.542313 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 314/620] 10.222.0.66 () {56 vars in 1103 bytes} [Mon Jul 18 02:15:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:33.544501 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b055f38e-a9a0-4bda-90dd-2c95a665fbe3 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:33.545009 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b055f38e-a9a0-4bda-90dd-2c95a665fbe3 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:33.545906 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b055f38e-a9a0-4bda-90dd-2c95a665fbe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:33.569708 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-b055f38e-a9a0-4bda-90dd-2c95a665fbe3 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:15:33.572110 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-b055f38e-a9a0-4bda-90dd-2c95a665fbe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:15:33.588136 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:33.616982 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316c0b7f-6f22-425b-81f7-f3243089278f None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:33.659110 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66f76b9-7fb1-4c12-805a-481c6fd32c34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:33.660041 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66f76b9-7fb1-4c12-805a-481c6fd32c34 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:33.660430 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66f76b9-7fb1-4c12-805a-481c6fd32c34 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:33.660794 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66f76b9-7fb1-4c12-805a-481c6fd32c34 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:33.663509 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 315/621] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:15:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 02:15:33.664939 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b055f38e-a9a0-4bda-90dd-2c95a665fbe3 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:33.682014 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66f76b9-7fb1-4c12-805a-481c6fd32c34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:33.683730 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66f76b9-7fb1-4c12-805a-481c6fd32c34 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:33.717823 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-f66f76b9-7fb1-4c12-805a-481c6fd32c34 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:15:33.717823 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:33.717823 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:33.717823 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:33.717823 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:33.717823 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:33.717823 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:33.717823 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:33.717823 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:33.717823 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:33.717823 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:33.717823 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:33.717823 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 02:15:33.717823 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 02:15:33.717823 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 02:15:33.718992 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 02:15:33.718992 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:33.718992 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:33.718992 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:33.718992 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:33.718992 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:33.718992 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:33.718992 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:33.718992 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:33.718992 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:33.718992 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:33.718992 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:33.718992 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:33.718992 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:33.718992 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:33.718992 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:33.718992 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:33.720117 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 02:15:33.720117 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 02:15:33.720117 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:15:33.720117 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:15:33.721114 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:33.747306 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 307/622] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 18 02:15:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:33.747306 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66f76b9-7fb1-4c12-805a-481c6fd32c34 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:33.791130 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b055f38e-a9a0-4bda-90dd-2c95a665fbe3 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:33.840478 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66f76b9-7fb1-4c12-805a-481c6fd32c34 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:33.842585 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf6d742-8432-4db5-8012-68cca85adb37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:33.844844 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf6d742-8432-4db5-8012-68cca85adb37 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:33.845288 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf6d742-8432-4db5-8012-68cca85adb37 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:33.845712 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf6d742-8432-4db5-8012-68cca85adb37 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:33.860596 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf6d742-8432-4db5-8012-68cca85adb37 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:33.862405 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf6d742-8432-4db5-8012-68cca85adb37 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:33.874871 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:33.881269 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527c9ae2-3240-4e3e-aa02-650eaac4adef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:33.882256 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527c9ae2-3240-4e3e-aa02-650eaac4adef None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:33.882589 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527c9ae2-3240-4e3e-aa02-650eaac4adef None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:33.882925 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527c9ae2-3240-4e3e-aa02-650eaac4adef None admin] PATH_INFO: `/v3/roles/service` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:33.889814 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-dcf6d742-8432-4db5-8012-68cca85adb37 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:15:33.889814 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:33.889814 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:33.889814 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:33.889814 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:33.889814 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:33.889814 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:33.889814 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:33.889814 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:33.889814 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:33.889814 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:33.889814 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:33.889814 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 02:15:33.889814 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 02:15:33.889814 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 02:15:33.890925 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 02:15:33.890925 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:33.890925 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:33.890925 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:33.890925 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:33.890925 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:33.890925 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:33.890925 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:33.890925 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:33.890925 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:33.890925 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:33.890925 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:33.890925 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:33.890925 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:33.890925 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:33.890925 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:33.890925 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:33.892033 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 02:15:33.892033 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 02:15:33.892033 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:15:33.892033 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:15:33.898766 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 316/623] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 18 02:15:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:33.899162 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf6d742-8432-4db5-8012-68cca85adb37 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:33.902736 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527c9ae2-3240-4e3e-aa02-650eaac4adef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 02:15:33.912332 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-527c9ae2-3240-4e3e-aa02-650eaac4adef None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 02:15:33.912332 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:33.912332 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:33.912332 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:33.912332 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:33.912332 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:33.912332 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:33.912332 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:33.912332 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:33.912332 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:33.912332 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:33.912332 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:33.912332 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 02:15:33.912332 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 02:15:33.912332 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 02:15:33.912332 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 02:15:33.913323 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 02:15:33.913323 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 02:15:33.913323 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 02:15:33.913323 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 02:15:33.913323 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:33.913323 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:33.913323 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:33.913323 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:33.913323 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:33.913323 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:33.913323 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:33.913323 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:33.913323 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:33.913323 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:33.913323 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:33.913323 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:33.914645 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:33.914645 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:33.914645 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:33.914645 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:33.914645 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 02:15:33.914645 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 02:15:33.914645 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 02:15:33.914645 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 02:15:33.914645 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 02:15:33.914645 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 02:15:33.914645 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 02:15:33.914645 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:15:33.918349 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 308/624] 10.222.0.66 () {56 vars in 1102 bytes} [Mon Jul 18 02:15:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:33.919811 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527c9ae2-3240-4e3e-aa02-650eaac4adef None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:34.022026 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:34.026229 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf6d742-8432-4db5-8012-68cca85adb37 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:34.037977 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527c9ae2-3240-4e3e-aa02-650eaac4adef None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:34.062989 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e41e194-6f39-440d-bdaa-f1dee1b216f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:34.063654 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e41e194-6f39-440d-bdaa-f1dee1b216f9 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:34.063978 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e41e194-6f39-440d-bdaa-f1dee1b216f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:34.064301 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e41e194-6f39-440d-bdaa-f1dee1b216f9 None admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:34.065023 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e41e194-6f39-440d-bdaa-f1dee1b216f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:34.065318 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e41e194-6f39-440d-bdaa-f1dee1b216f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:34.066787 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e41e194-6f39-440d-bdaa-f1dee1b216f9 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:34.079535 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f1b8ed-c472-4c90-88be-7d5f82ca9dfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:34.080823 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f1b8ed-c472-4c90-88be-7d5f82ca9dfc None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:34.081318 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f1b8ed-c472-4c90-88be-7d5f82ca9dfc None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:34.081812 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f1b8ed-c472-4c90-88be-7d5f82ca9dfc None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:34.082764 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f1b8ed-c472-4c90-88be-7d5f82ca9dfc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:34.083403 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f1b8ed-c472-4c90-88be-7d5f82ca9dfc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:34.084267 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 317/625] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 18 02:15:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:34.085193 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e41e194-6f39-440d-bdaa-f1dee1b216f9 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:34.086713 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f1b8ed-c472-4c90-88be-7d5f82ca9dfc None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:34.106115 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 309/626] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 18 02:15:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:34.107429 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f1b8ed-c472-4c90-88be-7d5f82ca9dfc None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:34.162557 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:34.191829 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e41e194-6f39-440d-bdaa-f1dee1b216f9 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:34.212649 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f1b8ed-c472-4c90-88be-7d5f82ca9dfc None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:34.230530 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae9554a-e551-441b-8b35-af14a4cc12e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:34.231338 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae9554a-e551-441b-8b35-af14a4cc12e1 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:34.232148 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae9554a-e551-441b-8b35-af14a4cc12e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:34.232436 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae9554a-e551-441b-8b35-af14a4cc12e1 None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:34.233239 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae9554a-e551-441b-8b35-af14a4cc12e1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:34.233547 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae9554a-e551-441b-8b35-af14a4cc12e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:34.234958 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae9554a-e551-441b-8b35-af14a4cc12e1 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:34.244390 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 318/627] 10.222.0.66 () {56 vars in 1103 bytes} [Mon Jul 18 02:15:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:34.245209 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae9554a-e551-441b-8b35-af14a4cc12e1 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:34.254973 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f04e47-30ee-497a-b953-e70416b9953a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:34.256107 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f04e47-30ee-497a-b953-e70416b9953a None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:34.256612 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f04e47-30ee-497a-b953-e70416b9953a None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:34.257105 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f04e47-30ee-497a-b953-e70416b9953a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:34.264797 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f04e47-30ee-497a-b953-e70416b9953a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:34.266544 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f04e47-30ee-497a-b953-e70416b9953a None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:34.273074 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:34.282366 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-09f04e47-30ee-497a-b953-e70416b9953a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 02:15:34.282366 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:34.282366 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:34.282366 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:34.282366 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:34.282366 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:34.282366 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:34.282366 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:34.282366 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:34.282366 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:34.282366 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:34.282366 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:34.282366 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 02:15:34.282366 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 02:15:34.282366 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 02:15:34.296139 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 02:15:34.296139 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:34.296139 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:34.296139 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:34.296139 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:34.296139 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:34.296139 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:34.296139 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:34.296139 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:34.296139 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:34.296139 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:34.296139 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:34.296139 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:34.296139 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:34.296139 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:34.296139 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:34.296139 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:34.297055 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 02:15:34.297055 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 02:15:34.297055 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 02:15:34.297055 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 02:15:34.297055 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 02:15:34.297055 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 02:15:34.297055 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 02:15:34.297055 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:15:34.297055 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 310/628] 10.222.0.66 () {56 vars in 1111 bytes} [Mon Jul 18 02:15:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:34.297055 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f04e47-30ee-497a-b953-e70416b9953a None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:34.357570 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae9554a-e551-441b-8b35-af14a4cc12e1 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:34.398643 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc21656-295b-4d2f-9cdb-0db834df2710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:34.399466 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc21656-295b-4d2f-9cdb-0db834df2710 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:34.399752 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc21656-295b-4d2f-9cdb-0db834df2710 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:34.400027 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc21656-295b-4d2f-9cdb-0db834df2710 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:34.402797 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f04e47-30ee-497a-b953-e70416b9953a None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:34.409738 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc21656-295b-4d2f-9cdb-0db834df2710 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:34.411394 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc21656-295b-4d2f-9cdb-0db834df2710 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:34.419449 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:34.428212 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-8bc21656-295b-4d2f-9cdb-0db834df2710 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 18 02:15:34.428212 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:34.428212 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:34.428212 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:34.428212 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:34.428212 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:34.428212 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:34.428212 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:34.428212 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:34.428212 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:34.428212 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:34.428212 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:34.428212 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 02:15:34.428212 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 02:15:34.428212 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 02:15:34.428212 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 02:15:34.429203 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:34.429203 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:34.429203 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 02:15:34.429203 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 02:15:34.429203 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 02:15:34.429203 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 02:15:34.429203 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:34.429203 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:34.429203 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:34.429203 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:34.429203 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:34.429203 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:34.429203 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:34.429203 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:34.429203 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:34.429203 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:34.429203 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:34.429203 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:34.430867 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:34.430867 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:34.430867 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 02:15:34.430867 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 02:15:34.430867 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 02:15:34.430867 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 02:15:34.430867 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 02:15:34.430867 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 02:15:34.430867 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 18 02:15:34.430867 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:15:34.434068 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 319/629] 10.222.0.66 () {56 vars in 1099 bytes} [Mon Jul 18 02:15:34 2022] GET /identity/v3/users/glance => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:34.435073 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc21656-295b-4d2f-9cdb-0db834df2710 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:34.449529 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fee547e-9437-4eea-844b-2a728a2aa89b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:34.451260 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fee547e-9437-4eea-844b-2a728a2aa89b None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:34.451779 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fee547e-9437-4eea-844b-2a728a2aa89b None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:34.452259 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fee547e-9437-4eea-844b-2a728a2aa89b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:34.463016 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fee547e-9437-4eea-844b-2a728a2aa89b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:34.464722 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fee547e-9437-4eea-844b-2a728a2aa89b None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:34.483782 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-5fee547e-9437-4eea-844b-2a728a2aa89b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:15:34.483782 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:34.483782 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:34.483782 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:34.483782 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:34.483782 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:34.483782 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:34.483782 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:34.483782 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:34.483782 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:34.483782 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:34.483782 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:34.483782 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 02:15:34.483782 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 02:15:34.483782 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 02:15:34.484748 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 02:15:34.484748 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:34.484748 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:34.484748 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:34.484748 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:34.484748 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:34.484748 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:34.484748 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:34.484748 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:34.484748 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:34.484748 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:34.484748 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:34.484748 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:34.484748 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:34.484748 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:34.484748 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:34.484748 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:34.485662 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 02:15:34.485662 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 02:15:34.485662 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:15:34.485662 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:15:34.487939 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 311/630] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 18 02:15:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:34.488927 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fee547e-9437-4eea-844b-2a728a2aa89b None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:34.530767 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:34.552793 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc21656-295b-4d2f-9cdb-0db834df2710 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:34.591873 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5185ee0d-79ba-41ca-b8a2-8b275e26aa03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:34.592540 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5185ee0d-79ba-41ca-b8a2-8b275e26aa03 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:34.592935 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5185ee0d-79ba-41ca-b8a2-8b275e26aa03 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:34.593181 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5185ee0d-79ba-41ca-b8a2-8b275e26aa03 None admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:34.594001 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5185ee0d-79ba-41ca-b8a2-8b275e26aa03 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:34.594560 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5185ee0d-79ba-41ca-b8a2-8b275e26aa03 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:34.595930 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5185ee0d-79ba-41ca-b8a2-8b275e26aa03 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:34.597645 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fee547e-9437-4eea-844b-2a728a2aa89b None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:34.620585 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 320/631] 10.222.0.66 () {56 vars in 1148 bytes} [Mon Jul 18 02:15:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:34.620585 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5185ee0d-79ba-41ca-b8a2-8b275e26aa03 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:34.642805 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80beaef5-4b3d-404c-9d6a-280e69d71e26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:34.643830 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80beaef5-4b3d-404c-9d6a-280e69d71e26 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:34.644325 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80beaef5-4b3d-404c-9d6a-280e69d71e26 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:34.644787 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80beaef5-4b3d-404c-9d6a-280e69d71e26 None admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:34.645830 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80beaef5-4b3d-404c-9d6a-280e69d71e26 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:34.646370 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80beaef5-4b3d-404c-9d6a-280e69d71e26 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:34.648026 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80beaef5-4b3d-404c-9d6a-280e69d71e26 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:34.666827 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 312/632] 10.222.0.66 () {56 vars in 1137 bytes} [Mon Jul 18 02:15:34 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:34.667936 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80beaef5-4b3d-404c-9d6a-280e69d71e26 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:34.694108 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:34.742417 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5185ee0d-79ba-41ca-b8a2-8b275e26aa03 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:34.776999 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80beaef5-4b3d-404c-9d6a-280e69d71e26 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:34.782258 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53dcfd6-bca8-43fb-b6e5-cee6bfdbfcd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:34.782865 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53dcfd6-bca8-43fb-b6e5-cee6bfdbfcd8 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:34.783148 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53dcfd6-bca8-43fb-b6e5-cee6bfdbfcd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:34.783422 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53dcfd6-bca8-43fb-b6e5-cee6bfdbfcd8 None admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:34.784142 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53dcfd6-bca8-43fb-b6e5-cee6bfdbfcd8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:34.784443 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53dcfd6-bca8-43fb-b6e5-cee6bfdbfcd8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:34.785811 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53dcfd6-bca8-43fb-b6e5-cee6bfdbfcd8 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:34.794838 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:34.798492 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 321/633] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 18 02:15:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:34.803661 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53dcfd6-bca8-43fb-b6e5-cee6bfdbfcd8 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:34.818145 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd355c3d-9b83-4212-91f4-31d4cb97856d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:34.819044 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd355c3d-9b83-4212-91f4-31d4cb97856d None admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:34.819326 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd355c3d-9b83-4212-91f4-31d4cb97856d None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:34.819814 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd355c3d-9b83-4212-91f4-31d4cb97856d None admin] PATH_INFO: `/v3/projects/76f7c0ae36ae49b09dcae4c3e9a875f4/users/b4300622cbd347dd8f5227bb960e98d0/roles/eea94daf3312421cb12aa8abfa73160e` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:34.835685 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd355c3d-9b83-4212-91f4-31d4cb97856d None admin] RBAC: Authorizing `identity:create_grant(project_id=76f7c0ae36ae49b09dcae4c3e9a875f4, user_id=b4300622cbd347dd8f5227bb960e98d0, role_id=eea94daf3312421cb12aa8abfa73160e)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:34.837185 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd355c3d-9b83-4212-91f4-31d4cb97856d None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:34.904836 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53dcfd6-bca8-43fb-b6e5-cee6bfdbfcd8 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:34.940202 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 313/634] 10.222.0.66 () {58 vars in 1439 bytes} [Mon Jul 18 02:15:34 2022] PUT /identity/v3/projects/76f7c0ae36ae49b09dcae4c3e9a875f4/users/b4300622cbd347dd8f5227bb960e98d0/roles/eea94daf3312421cb12aa8abfa73160e => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:15:34.940202 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc515c96-c86b-4ec5-8634-68e346c7b731 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:34.940202 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc515c96-c86b-4ec5-8634-68e346c7b731 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:34.940202 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc515c96-c86b-4ec5-8634-68e346c7b731 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:34.940202 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 314/635] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:15:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:15:34.940202 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc515c96-c86b-4ec5-8634-68e346c7b731 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:34.945847 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368de5ee-3fb5-4ac6-af70-a546575baf45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:34.950871 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368de5ee-3fb5-4ac6-af70-a546575baf45 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:34.950871 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368de5ee-3fb5-4ac6-af70-a546575baf45 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:34.950871 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368de5ee-3fb5-4ac6-af70-a546575baf45 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:35.026950 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:35.032712 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368de5ee-3fb5-4ac6-af70-a546575baf45 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:35.034169 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368de5ee-3fb5-4ac6-af70-a546575baf45 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:35.041492 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:35.088494 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc515c96-c86b-4ec5-8634-68e346c7b731 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:35.136028 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d64bae2-9c50-4c4a-969e-6125cdb28bdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:35.136869 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d64bae2-9c50-4c4a-969e-6125cdb28bdf None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:35.137153 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d64bae2-9c50-4c4a-969e-6125cdb28bdf None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:35.137430 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d64bae2-9c50-4c4a-969e-6125cdb28bdf None admin] PATH_INFO: `/v3/users/nova` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:35.145924 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d64bae2-9c50-4c4a-969e-6125cdb28bdf None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:35.147501 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d64bae2-9c50-4c4a-969e-6125cdb28bdf None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:35.155179 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 322/636] 10.222.0.66 () {58 vars in 1282 bytes} [Mon Jul 18 02:15:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 02:15:35.155179 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1b50e6-d9b9-4865-ace5-510c57d0a3ab None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:35.155179 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1b50e6-d9b9-4865-ace5-510c57d0a3ab None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:35.155179 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1b50e6-d9b9-4865-ace5-510c57d0a3ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:35.163105 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-4d64bae2-9c50-4c4a-969e-6125cdb28bdf None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 18 02:15:35.163105 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:35.163105 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:35.163105 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:35.163105 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:35.163105 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:35.163105 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:35.163105 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:35.163105 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:35.163105 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:35.163105 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:35.163105 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:35.163105 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 02:15:35.163105 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 02:15:35.163105 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 02:15:35.163105 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 02:15:35.164084 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:35.164084 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:35.164084 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 02:15:35.164084 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 02:15:35.164084 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 02:15:35.164084 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 02:15:35.164084 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:35.164084 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:35.164084 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:35.164084 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:35.164084 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:35.164084 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:35.164084 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:35.164084 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:35.164084 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:35.164084 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:35.164084 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:35.164084 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:35.165047 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:35.165047 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:35.165047 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 02:15:35.165047 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 02:15:35.165047 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 02:15:35.165047 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 02:15:35.165047 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 02:15:35.165047 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 02:15:35.165047 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 18 02:15:35.165047 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:15:35.165047 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 315/637] 10.222.0.66 () {56 vars in 1093 bytes} [Mon Jul 18 02:15:34 2022] GET /identity/v3/users/nova => generated 82 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:35.166259 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce52701f-22bc-4178-a0db-80baf8fad04a None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:35.166259 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce52701f-22bc-4178-a0db-80baf8fad04a None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:35.166464 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce52701f-22bc-4178-a0db-80baf8fad04a None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:35.171440 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 316/638] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:15:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:15:35.172069 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce52701f-22bc-4178-a0db-80baf8fad04a None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:35.175647 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-6e1b50e6-d9b9-4865-ace5-510c57d0a3ab None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:15:35.177954 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-6e1b50e6-d9b9-4865-ace5-510c57d0a3ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:15:35.251038 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 323/639] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:15:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 02:15:35.251850 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1b50e6-d9b9-4865-ace5-510c57d0a3ab None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:35.286543 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:35.298292 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce52701f-22bc-4178-a0db-80baf8fad04a None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:35.329775 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:35.339024 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f73099-fecc-46b0-8b9a-7863f7fc704b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:35.343907 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f73099-fecc-46b0-8b9a-7863f7fc704b None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:35.343907 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f73099-fecc-46b0-8b9a-7863f7fc704b None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:35.343907 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f73099-fecc-46b0-8b9a-7863f7fc704b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:35.351071 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f73099-fecc-46b0-8b9a-7863f7fc704b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:35.352547 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f73099-fecc-46b0-8b9a-7863f7fc704b None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:35.361423 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1b50e6-d9b9-4865-ace5-510c57d0a3ab None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:35.373870 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-71f73099-fecc-46b0-8b9a-7863f7fc704b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:15:35.373870 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:35.373870 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:35.373870 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:35.373870 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:35.373870 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:35.373870 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:35.373870 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:35.373870 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:35.373870 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:35.373870 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:35.373870 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:35.373870 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 02:15:35.373870 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 02:15:35.373870 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 02:15:35.374943 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 02:15:35.374943 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:35.374943 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:35.374943 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:35.374943 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:35.374943 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:35.374943 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:35.374943 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:35.374943 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:35.374943 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:35.374943 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:35.374943 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:35.374943 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:35.374943 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:35.374943 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:35.374943 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:35.374943 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:35.376051 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 02:15:35.376051 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 02:15:35.376051 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:15:35.376051 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:15:35.377625 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 317/640] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 18 02:15:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:35.379046 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3deede-3f8e-43ce-b0c3-f010dc77a33c None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:35.379332 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3deede-3f8e-43ce-b0c3-f010dc77a33c None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:35.379610 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3deede-3f8e-43ce-b0c3-f010dc77a33c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:35.396373 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-2a3deede-3f8e-43ce-b0c3-f010dc77a33c None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:15:35.397568 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-2a3deede-3f8e-43ce-b0c3-f010dc77a33c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:15:35.404943 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65dd277-6579-44af-b260-49d2a62763ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:35.405852 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65dd277-6579-44af-b260-49d2a62763ff None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:35.406366 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65dd277-6579-44af-b260-49d2a62763ff None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:35.406813 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65dd277-6579-44af-b260-49d2a62763ff None admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:35.407881 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65dd277-6579-44af-b260-49d2a62763ff None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:35.408416 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65dd277-6579-44af-b260-49d2a62763ff None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:35.410017 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65dd277-6579-44af-b260-49d2a62763ff None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:35.431986 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 324/641] 10.222.0.66 () {56 vars in 1133 bytes} [Mon Jul 18 02:15:35 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:35.433241 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65dd277-6579-44af-b260-49d2a62763ff None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:35.468581 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 318/642] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:15:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 02:15:35.469734 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3deede-3f8e-43ce-b0c3-f010dc77a33c None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:35.547995 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:35.559831 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65dd277-6579-44af-b260-49d2a62763ff None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:35.577834 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3deede-3f8e-43ce-b0c3-f010dc77a33c None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:35.593643 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:35.607859 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e57436-0889-4ef5-957d-80f4fd99308d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:35.608670 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e57436-0889-4ef5-957d-80f4fd99308d None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:35.608958 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e57436-0889-4ef5-957d-80f4fd99308d None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:35.609242 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e57436-0889-4ef5-957d-80f4fd99308d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:35.617815 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b469bdbc-adf2-4ccc-8ed3-aa489b3fdd14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:35.618528 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b469bdbc-adf2-4ccc-8ed3-aa489b3fdd14 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:35.618813 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b469bdbc-adf2-4ccc-8ed3-aa489b3fdd14 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:35.619090 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b469bdbc-adf2-4ccc-8ed3-aa489b3fdd14 None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:35.619815 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b469bdbc-adf2-4ccc-8ed3-aa489b3fdd14 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:35.620125 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b469bdbc-adf2-4ccc-8ed3-aa489b3fdd14 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:35.620241 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e57436-0889-4ef5-957d-80f4fd99308d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 02:15:35.621520 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b469bdbc-adf2-4ccc-8ed3-aa489b3fdd14 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:35.629095 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-91e57436-0889-4ef5-957d-80f4fd99308d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 02:15:35.629095 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:35.629095 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:35.629095 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:35.629095 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:35.629095 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:35.629095 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:35.629095 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:35.629095 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:35.629095 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:35.629095 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:35.629095 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:35.629095 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 02:15:35.629095 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 02:15:35.629095 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 02:15:35.629095 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 02:15:35.630951 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 02:15:35.630951 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 02:15:35.630951 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 02:15:35.630951 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 02:15:35.630951 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:35.630951 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:35.630951 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:35.630951 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:35.630951 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:35.630951 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:35.630951 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:35.630951 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:35.630951 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:35.630951 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:35.630951 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:35.630951 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:35.631886 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:35.631886 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:35.631886 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:35.631886 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:35.631886 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 02:15:35.631886 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 02:15:35.631886 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 02:15:35.631886 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 02:15:35.631886 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 02:15:35.631886 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 02:15:35.631886 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 02:15:35.631886 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:15:35.633039 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 325/643] 10.222.0.66 () {56 vars in 1102 bytes} [Mon Jul 18 02:15:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:35.633358 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e57436-0889-4ef5-957d-80f4fd99308d None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:35.636478 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 319/644] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 18 02:15:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:35.637756 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b469bdbc-adf2-4ccc-8ed3-aa489b3fdd14 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:35.744652 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b469bdbc-adf2-4ccc-8ed3-aa489b3fdd14 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:35.745226 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e57436-0889-4ef5-957d-80f4fd99308d None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:35.780454 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e983ad5d-46e8-4fbb-92e3-e36c113f45f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:35.780895 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345a912c-b7e6-49d9-a098-f909c11166f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:35.781357 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e983ad5d-46e8-4fbb-92e3-e36c113f45f0 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:35.781357 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e983ad5d-46e8-4fbb-92e3-e36c113f45f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:35.781637 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e983ad5d-46e8-4fbb-92e3-e36c113f45f0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:35.781738 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345a912c-b7e6-49d9-a098-f909c11166f3 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:35.781961 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345a912c-b7e6-49d9-a098-f909c11166f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:35.782268 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345a912c-b7e6-49d9-a098-f909c11166f3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:35.793394 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345a912c-b7e6-49d9-a098-f909c11166f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 02:15:35.800261 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:35.801750 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-345a912c-b7e6-49d9-a098-f909c11166f3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 02:15:35.801750 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:35.801750 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:35.801750 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:35.801750 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:35.801750 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:35.801750 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:35.801750 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:35.801750 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:35.801750 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:35.801750 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:35.801750 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:35.801750 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 02:15:35.801750 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 02:15:35.801750 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 02:15:35.801750 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 02:15:35.802728 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 02:15:35.802728 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 02:15:35.802728 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 02:15:35.802728 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 02:15:35.802728 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:35.802728 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:35.802728 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:35.802728 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:35.802728 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:35.802728 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:35.802728 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:35.802728 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:35.802728 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:35.802728 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:35.802728 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:35.802728 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:35.803737 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:35.803737 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:35.803737 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:35.803737 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:35.803737 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 02:15:35.803737 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 02:15:35.803737 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 02:15:35.803737 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 02:15:35.803737 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 02:15:35.803737 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 02:15:35.803737 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 02:15:35.803737 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:15:35.803737 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 320/645] 10.222.0.66 () {56 vars in 1102 bytes} [Mon Jul 18 02:15:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:35.804520 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345a912c-b7e6-49d9-a098-f909c11166f3 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:35.843749 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:35.858184 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e983ad5d-46e8-4fbb-92e3-e36c113f45f0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:35.859725 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e983ad5d-46e8-4fbb-92e3-e36c113f45f0 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:35.912784 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345a912c-b7e6-49d9-a098-f909c11166f3 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:35.950652 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7280d00f-44dd-4bfe-8fd0-c54881f2f7c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:35.951243 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7280d00f-44dd-4bfe-8fd0-c54881f2f7c6 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:35.951828 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7280d00f-44dd-4bfe-8fd0-c54881f2f7c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:35.951941 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7280d00f-44dd-4bfe-8fd0-c54881f2f7c6 None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:35.952647 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7280d00f-44dd-4bfe-8fd0-c54881f2f7c6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:35.952955 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7280d00f-44dd-4bfe-8fd0-c54881f2f7c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:35.954468 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7280d00f-44dd-4bfe-8fd0-c54881f2f7c6 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:35.965353 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 326/646] 10.222.0.66 () {58 vars in 1282 bytes} [Mon Jul 18 02:15:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 02:15:35.967201 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e983ad5d-46e8-4fbb-92e3-e36c113f45f0 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:35.967656 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 321/647] 10.222.0.66 () {56 vars in 1103 bytes} [Mon Jul 18 02:15:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:35.968513 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7280d00f-44dd-4bfe-8fd0-c54881f2f7c6 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:36.021346 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:36.072860 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:36.074578 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e983ad5d-46e8-4fbb-92e3-e36c113f45f0 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:36.079095 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7280d00f-44dd-4bfe-8fd0-c54881f2f7c6 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:36.113908 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9579dd-f898-4970-a982-8d6b5e96a57c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:36.114723 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9579dd-f898-4970-a982-8d6b5e96a57c None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:36.115010 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9579dd-f898-4970-a982-8d6b5e96a57c None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:36.115290 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9579dd-f898-4970-a982-8d6b5e96a57c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:36.116972 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8031c652-bf14-4780-9900-ca8f64275234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:36.117640 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8031c652-bf14-4780-9900-ca8f64275234 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:36.117925 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8031c652-bf14-4780-9900-ca8f64275234 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:36.118265 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8031c652-bf14-4780-9900-ca8f64275234 None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:36.119020 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8031c652-bf14-4780-9900-ca8f64275234 None admin] RBAC: Adding query filter params (name=service) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:36.119328 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8031c652-bf14-4780-9900-ca8f64275234 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:36.120724 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8031c652-bf14-4780-9900-ca8f64275234 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:36.123175 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9579dd-f898-4970-a982-8d6b5e96a57c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:36.124613 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9579dd-f898-4970-a982-8d6b5e96a57c None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:36.134001 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 322/648] 10.222.0.66 () {56 vars in 1103 bytes} [Mon Jul 18 02:15:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:36.134001 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8031c652-bf14-4780-9900-ca8f64275234 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:36.143302 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-0b9579dd-f898-4970-a982-8d6b5e96a57c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 02:15:36.143302 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:36.143302 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:36.143302 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:36.143302 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:36.143302 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:36.143302 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:36.143302 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:36.143302 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:36.143302 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:36.143302 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:36.143302 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:36.143302 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 02:15:36.143302 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 02:15:36.143302 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 02:15:36.144395 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 02:15:36.144395 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:36.144395 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:36.144395 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:36.144395 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:36.144395 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:36.144395 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:36.144395 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:36.144395 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:36.144395 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:36.144395 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:36.144395 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:36.144395 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:36.144395 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:36.144395 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:36.144395 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:36.144395 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:36.145339 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 02:15:36.145339 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 02:15:36.145339 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 02:15:36.145339 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 02:15:36.145339 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 02:15:36.145339 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 02:15:36.145339 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 02:15:36.145339 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:15:36.145339 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 327/649] 10.222.0.66 () {56 vars in 1111 bytes} [Mon Jul 18 02:15:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:36.147132 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9579dd-f898-4970-a982-8d6b5e96a57c None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:36.241875 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8031c652-bf14-4780-9900-ca8f64275234 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:36.257048 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9579dd-f898-4970-a982-8d6b5e96a57c None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:36.266960 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:36.276653 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d95a31-f68a-4e06-878a-3a6d9d10d56a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:36.280898 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d95a31-f68a-4e06-878a-3a6d9d10d56a None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:36.280898 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d95a31-f68a-4e06-878a-3a6d9d10d56a None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:36.280898 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d95a31-f68a-4e06-878a-3a6d9d10d56a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:36.289748 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d95a31-f68a-4e06-878a-3a6d9d10d56a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:36.291256 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d95a31-f68a-4e06-878a-3a6d9d10d56a None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:36.293488 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caeef6a-bacb-403c-bc29-b9a515809455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:36.295168 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caeef6a-bacb-403c-bc29-b9a515809455 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:36.295636 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caeef6a-bacb-403c-bc29-b9a515809455 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:36.296089 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caeef6a-bacb-403c-bc29-b9a515809455 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:36.308306 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1caeef6a-bacb-403c-bc29-b9a515809455 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:36.309387 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-97d95a31-f68a-4e06-878a-3a6d9d10d56a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:15:36.309387 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:36.309387 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:36.309387 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:36.309387 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:36.309387 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:36.309387 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:36.309387 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:36.309387 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:36.309387 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:36.309387 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:36.309387 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:36.309387 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 02:15:36.309387 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 02:15:36.309387 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 02:15:36.310317 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 02:15:36.310317 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:36.310317 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:36.310317 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:36.310317 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:36.310317 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:36.310317 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:36.310317 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:36.310317 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:36.310317 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:36.310317 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:36.310317 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:36.310317 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:36.310317 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:36.310317 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:36.310317 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:36.310317 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:36.311448 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 02:15:36.311448 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 02:15:36.311448 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:15:36.311448 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:15:36.312408 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1caeef6a-bacb-403c-bc29-b9a515809455 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:36.315072 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 323/650] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 18 02:15:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:36.315865 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d95a31-f68a-4e06-878a-3a6d9d10d56a None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:36.329196 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:36.332407 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-1caeef6a-bacb-403c-bc29-b9a515809455 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:15:36.332407 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:36.332407 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:36.332407 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:36.332407 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:36.332407 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:36.332407 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:36.332407 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:36.332407 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:36.332407 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:36.332407 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:36.332407 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:36.332407 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 02:15:36.332407 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 02:15:36.332407 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 02:15:36.333467 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 02:15:36.333467 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:36.333467 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:36.333467 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:36.333467 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:36.333467 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:36.333467 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:36.333467 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:36.333467 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:36.333467 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:36.333467 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:36.333467 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:36.333467 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:36.333467 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:36.333467 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:36.333467 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:36.333467 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:36.334555 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 02:15:36.334555 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 02:15:36.334555 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:15:36.334555 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:15:36.335526 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 328/651] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 18 02:15:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:36.337109 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caeef6a-bacb-403c-bc29-b9a515809455 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:36.428016 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d95a31-f68a-4e06-878a-3a6d9d10d56a None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:36.444530 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caeef6a-bacb-403c-bc29-b9a515809455 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:36.463935 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2f89c3-81ca-4ff7-82ab-78f8290d502a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:36.464740 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2f89c3-81ca-4ff7-82ab-78f8290d502a None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:36.465028 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2f89c3-81ca-4ff7-82ab-78f8290d502a None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:36.465312 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2f89c3-81ca-4ff7-82ab-78f8290d502a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:36.477578 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2f89c3-81ca-4ff7-82ab-78f8290d502a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:36.479063 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2f89c3-81ca-4ff7-82ab-78f8290d502a None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:36.480954 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c352d58-e6c3-471e-8ee9-8a6127d95aa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:36.481998 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c352d58-e6c3-471e-8ee9-8a6127d95aa7 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:36.482498 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c352d58-e6c3-471e-8ee9-8a6127d95aa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:36.482952 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c352d58-e6c3-471e-8ee9-8a6127d95aa7 None admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:36.483911 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c352d58-e6c3-471e-8ee9-8a6127d95aa7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:36.484488 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c352d58-e6c3-471e-8ee9-8a6127d95aa7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:36.486245 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c352d58-e6c3-471e-8ee9-8a6127d95aa7 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:36.499205 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-0e2f89c3-81ca-4ff7-82ab-78f8290d502a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:15:36.499205 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:36.499205 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:36.499205 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:36.499205 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:36.499205 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:36.499205 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:36.499205 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:36.499205 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:36.499205 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:36.499205 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:36.499205 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:36.499205 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 02:15:36.499205 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 02:15:36.499205 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 02:15:36.500239 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 02:15:36.500239 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:36.500239 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:36.500239 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:36.500239 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:36.500239 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:36.500239 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:36.500239 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:36.500239 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:36.500239 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:36.500239 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:36.500239 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:36.500239 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:36.500239 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:36.500239 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:36.500239 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:36.500239 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:36.501150 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 02:15:36.501150 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 02:15:36.501150 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:15:36.501150 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:15:36.501150 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 324/652] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 18 02:15:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:36.501445 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2f89c3-81ca-4ff7-82ab-78f8290d502a None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:36.503547 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 329/653] 10.222.0.66 () {56 vars in 1148 bytes} [Mon Jul 18 02:15:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:36.504555 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c352d58-e6c3-471e-8ee9-8a6127d95aa7 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:36.527627 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:36.577635 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:36.625786 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2f89c3-81ca-4ff7-82ab-78f8290d502a None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:36.625972 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c352d58-e6c3-471e-8ee9-8a6127d95aa7 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:36.664275 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3991dae-aac2-461e-8bb0-6bf19d2c46d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:36.664629 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33741027-ea20-435e-bcea-bf13f59c53ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:36.664946 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3991dae-aac2-461e-8bb0-6bf19d2c46d2 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:36.665200 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33741027-ea20-435e-bcea-bf13f59c53ce None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:36.665300 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3991dae-aac2-461e-8bb0-6bf19d2c46d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:36.665484 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33741027-ea20-435e-bcea-bf13f59c53ce None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:36.665582 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3991dae-aac2-461e-8bb0-6bf19d2c46d2 None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:36.665760 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33741027-ea20-435e-bcea-bf13f59c53ce None admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:36.666231 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3991dae-aac2-461e-8bb0-6bf19d2c46d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:36.666516 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33741027-ea20-435e-bcea-bf13f59c53ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:36.666628 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3991dae-aac2-461e-8bb0-6bf19d2c46d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:36.666824 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33741027-ea20-435e-bcea-bf13f59c53ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:36.668177 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33741027-ea20-435e-bcea-bf13f59c53ce None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:36.668334 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3991dae-aac2-461e-8bb0-6bf19d2c46d2 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:36.680760 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 330/654] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 18 02:15:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:36.681551 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33741027-ea20-435e-bcea-bf13f59c53ce None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:36.696835 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 325/655] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 18 02:15:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:36.696835 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3991dae-aac2-461e-8bb0-6bf19d2c46d2 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:36.788838 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:36.812761 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3991dae-aac2-461e-8bb0-6bf19d2c46d2 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:36.816086 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33741027-ea20-435e-bcea-bf13f59c53ce None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:36.839563 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:36.853072 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dce0f00-ed3d-4e78-85ea-ea5451cb32a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:36.853740 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dce0f00-ed3d-4e78-85ea-ea5451cb32a9 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:36.854031 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dce0f00-ed3d-4e78-85ea-ea5451cb32a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:36.854338 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dce0f00-ed3d-4e78-85ea-ea5451cb32a9 None admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:36.855038 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56d304f-c7ad-4e51-8de3-20d8532caa82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:36.855162 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dce0f00-ed3d-4e78-85ea-ea5451cb32a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:36.855361 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dce0f00-ed3d-4e78-85ea-ea5451cb32a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:36.856765 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56d304f-c7ad-4e51-8de3-20d8532caa82 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:36.857526 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dce0f00-ed3d-4e78-85ea-ea5451cb32a9 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:36.857905 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56d304f-c7ad-4e51-8de3-20d8532caa82 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:36.858449 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56d304f-c7ad-4e51-8de3-20d8532caa82 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:36.859810 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56d304f-c7ad-4e51-8de3-20d8532caa82 None admin] RBAC: Adding query filter params (role.id=eea94daf3312421cb12aa8abfa73160e, scope.project.id=76f7c0ae36ae49b09dcae4c3e9a875f4, user.id=e1391b81e7234bc18e22b9e84f231256) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:36.860890 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56d304f-c7ad-4e51-8de3-20d8532caa82 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:36.862698 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56d304f-c7ad-4e51-8de3-20d8532caa82 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:36.878117 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 326/656] 10.222.0.66 () {56 vars in 1374 bytes} [Mon Jul 18 02:15:36 2022] GET /identity/v3/role_assignments?user.id=e1391b81e7234bc18e22b9e84f231256&scope.project.id=76f7c0ae36ae49b09dcae4c3e9a875f4&role.id=eea94daf3312421cb12aa8abfa73160e => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:36.878117 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56d304f-c7ad-4e51-8de3-20d8532caa82 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:36.878407 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 331/657] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 18 02:15:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:36.878407 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dce0f00-ed3d-4e78-85ea-ea5451cb32a9 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:37.012237 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56d304f-c7ad-4e51-8de3-20d8532caa82 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:37.023172 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dce0f00-ed3d-4e78-85ea-ea5451cb32a9 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:37.055928 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ce1e3b-d93a-4b3c-a261-b3d662e71d76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:37.056738 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ce1e3b-d93a-4b3c-a261-b3d662e71d76 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:37.057023 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ce1e3b-d93a-4b3c-a261-b3d662e71d76 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:37.057298 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ce1e3b-d93a-4b3c-a261-b3d662e71d76 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:37.065466 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ce1e3b-d93a-4b3c-a261-b3d662e71d76 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:37.066973 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ce1e3b-d93a-4b3c-a261-b3d662e71d76 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:37.072445 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9840de-c58b-4ecc-879e-8a1b3f10a9b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:37.073254 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9840de-c58b-4ecc-879e-8a1b3f10a9b9 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:37.073539 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9840de-c58b-4ecc-879e-8a1b3f10a9b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:37.073818 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9840de-c58b-4ecc-879e-8a1b3f10a9b9 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:37.082011 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9840de-c58b-4ecc-879e-8a1b3f10a9b9 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:37.083504 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9840de-c58b-4ecc-879e-8a1b3f10a9b9 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:37.083747 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-69ce1e3b-d93a-4b3c-a261-b3d662e71d76 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 02:15:37.083747 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:37.083747 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:37.083747 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:37.083747 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:37.083747 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:37.083747 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:37.083747 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:37.083747 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:37.083747 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:37.083747 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:37.083747 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:37.083747 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 02:15:37.083747 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 02:15:37.083747 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 02:15:37.084635 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 02:15:37.084635 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:37.084635 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:37.084635 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:37.084635 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:37.084635 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:37.084635 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:37.084635 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:37.084635 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:37.084635 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:37.084635 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:37.084635 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:37.084635 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:37.084635 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:37.084635 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:37.084635 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:37.084635 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:37.085543 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 02:15:37.085543 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 02:15:37.085543 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 02:15:37.085543 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 02:15:37.085543 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 02:15:37.085543 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 02:15:37.085543 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 02:15:37.085543 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:15:37.085543 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 327/658] 10.222.0.66 () {56 vars in 1111 bytes} [Mon Jul 18 02:15:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:37.086047 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ce1e3b-d93a-4b3c-a261-b3d662e71d76 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:37.090350 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:37.104515 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-8d9840de-c58b-4ecc-879e-8a1b3f10a9b9 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 18 02:15:37.104515 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:37.104515 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:37.104515 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:37.104515 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:37.104515 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:37.104515 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:37.104515 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:37.104515 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:37.104515 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:37.104515 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:37.104515 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:37.104515 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 02:15:37.104515 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 02:15:37.104515 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 02:15:37.104515 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 02:15:37.107289 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:37.107289 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:37.107289 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 02:15:37.107289 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 02:15:37.107289 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 02:15:37.107289 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 02:15:37.107289 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:37.107289 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:37.107289 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:37.107289 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:37.107289 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:37.107289 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:37.107289 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:37.107289 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:37.107289 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:37.107289 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:37.107289 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:37.107289 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:37.109316 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:37.109316 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:37.109316 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 02:15:37.109316 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 02:15:37.109316 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 02:15:37.109316 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 02:15:37.109316 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 02:15:37.109316 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 02:15:37.109316 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 18 02:15:37.109316 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:15:37.110640 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 332/659] 10.222.0.66 () {56 vars in 1099 bytes} [Mon Jul 18 02:15:36 2022] GET /identity/v3/users/cinder => generated 84 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:37.112061 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13a6835-4d23-43e1-b442-763550e7299c None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:37.112353 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13a6835-4d23-43e1-b442-763550e7299c None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:37.112653 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13a6835-4d23-43e1-b442-763550e7299c None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:37.114332 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 333/660] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:15:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:15:37.114790 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13a6835-4d23-43e1-b442-763550e7299c None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:37.144321 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:37.219617 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ce1e3b-d93a-4b3c-a261-b3d662e71d76 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:37.239745 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13a6835-4d23-43e1-b442-763550e7299c None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:37.259275 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2604445a-7f97-42ff-a921-ea1679b9c756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:37.260402 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2604445a-7f97-42ff-a921-ea1679b9c756 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:37.260925 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2604445a-7f97-42ff-a921-ea1679b9c756 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:37.261400 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2604445a-7f97-42ff-a921-ea1679b9c756 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:37.272912 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2604445a-7f97-42ff-a921-ea1679b9c756 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:37.274419 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2604445a-7f97-42ff-a921-ea1679b9c756 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:37.280725 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e886091f-2c64-403b-9827-8bcc37287c33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:37.281615 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e886091f-2c64-403b-9827-8bcc37287c33 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:37.282428 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e886091f-2c64-403b-9827-8bcc37287c33 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:37.282913 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e886091f-2c64-403b-9827-8bcc37287c33 None admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:37.284125 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e886091f-2c64-403b-9827-8bcc37287c33 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:37.284647 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e886091f-2c64-403b-9827-8bcc37287c33 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:37.286366 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e886091f-2c64-403b-9827-8bcc37287c33 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:37.290811 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-2604445a-7f97-42ff-a921-ea1679b9c756 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 18 02:15:37.290811 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:37.290811 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:37.290811 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:37.290811 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:37.290811 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:37.290811 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:37.290811 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:37.290811 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:37.290811 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:37.290811 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:37.290811 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:37.290811 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 02:15:37.290811 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 02:15:37.290811 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 02:15:37.290811 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 02:15:37.291807 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:37.291807 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:37.291807 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 02:15:37.291807 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 02:15:37.291807 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 02:15:37.291807 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 02:15:37.291807 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:37.291807 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:37.291807 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:37.291807 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:37.291807 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:37.291807 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:37.291807 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:37.291807 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:37.291807 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:37.291807 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:37.291807 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:37.291807 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:37.292773 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:37.292773 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:37.292773 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 02:15:37.292773 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 02:15:37.292773 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 02:15:37.292773 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 02:15:37.292773 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 02:15:37.292773 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 02:15:37.292773 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 18 02:15:37.292773 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:15:37.292773 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 328/661] 10.222.0.66 () {56 vars in 1102 bytes} [Mon Jul 18 02:15:37 2022] GET /identity/v3/users/neutron => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:37.292773 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2604445a-7f97-42ff-a921-ea1679b9c756 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:37.304198 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 334/662] 10.222.0.66 () {56 vars in 1148 bytes} [Mon Jul 18 02:15:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:37.306767 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432560f8-b776-4f05-9592-571a8929b25b None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:37.307294 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432560f8-b776-4f05-9592-571a8929b25b None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:37.307723 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432560f8-b776-4f05-9592-571a8929b25b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:37.333970 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-432560f8-b776-4f05-9592-571a8929b25b None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:15:37.336581 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-432560f8-b776-4f05-9592-571a8929b25b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:15:37.404883 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:37.410526 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2604445a-7f97-42ff-a921-ea1679b9c756 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:37.422342 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 335/663] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:15:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 02:15:37.422342 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432560f8-b776-4f05-9592-571a8929b25b None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:37.429528 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:37.458594 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f383338b-9928-4270-adaa-fcba8ccb0932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:37.459317 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f383338b-9928-4270-adaa-fcba8ccb0932 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:37.459602 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f383338b-9928-4270-adaa-fcba8ccb0932 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:37.459819 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f383338b-9928-4270-adaa-fcba8ccb0932 None admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:37.460669 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f383338b-9928-4270-adaa-fcba8ccb0932 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:37.460978 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f383338b-9928-4270-adaa-fcba8ccb0932 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:37.462446 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f383338b-9928-4270-adaa-fcba8ccb0932 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:37.484811 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 329/664] 10.222.0.66 () {56 vars in 1137 bytes} [Mon Jul 18 02:15:37 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:37.485671 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f383338b-9928-4270-adaa-fcba8ccb0932 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:37.541165 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432560f8-b776-4f05-9592-571a8929b25b None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:37.580500 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d975f12a-c5a9-4635-a9a0-ee515967a579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:37.581254 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d975f12a-c5a9-4635-a9a0-ee515967a579 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:37.581754 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d975f12a-c5a9-4635-a9a0-ee515967a579 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:37.582669 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d975f12a-c5a9-4635-a9a0-ee515967a579 None admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:37.583383 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d975f12a-c5a9-4635-a9a0-ee515967a579 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:37.583696 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d975f12a-c5a9-4635-a9a0-ee515967a579 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:37.585130 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d975f12a-c5a9-4635-a9a0-ee515967a579 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:37.610386 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 336/665] 10.222.0.66 () {56 vars in 1139 bytes} [Mon Jul 18 02:15:37 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:37.611209 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d975f12a-c5a9-4635-a9a0-ee515967a579 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:37.617136 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f383338b-9928-4270-adaa-fcba8ccb0932 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:37.654425 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fb20ea-6d5e-4568-8e9a-9deee0cf669c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:37.655515 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fb20ea-6d5e-4568-8e9a-9deee0cf669c None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:37.655797 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fb20ea-6d5e-4568-8e9a-9deee0cf669c None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:37.656075 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fb20ea-6d5e-4568-8e9a-9deee0cf669c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:37.656905 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fb20ea-6d5e-4568-8e9a-9deee0cf669c None admin] RBAC: Adding query filter params (role.id=eea94daf3312421cb12aa8abfa73160e, scope.project.id=76f7c0ae36ae49b09dcae4c3e9a875f4, user.id=80b138831bc54c7b986d50092d62d4af) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:37.657219 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fb20ea-6d5e-4568-8e9a-9deee0cf669c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:37.658709 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fb20ea-6d5e-4568-8e9a-9deee0cf669c None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:37.665814 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:37.674526 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 330/666] 10.222.0.66 () {56 vars in 1374 bytes} [Mon Jul 18 02:15:37 2022] GET /identity/v3/role_assignments?user.id=80b138831bc54c7b986d50092d62d4af&scope.project.id=76f7c0ae36ae49b09dcae4c3e9a875f4&role.id=eea94daf3312421cb12aa8abfa73160e => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:37.676542 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fb20ea-6d5e-4568-8e9a-9deee0cf669c None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:37.728518 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:37.731881 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d975f12a-c5a9-4635-a9a0-ee515967a579 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:37.784001 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a733a90-3637-4b7c-9240-9356f34a5255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:37.785152 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a733a90-3637-4b7c-9240-9356f34a5255 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:37.786250 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a733a90-3637-4b7c-9240-9356f34a5255 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:37.786788 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a733a90-3637-4b7c-9240-9356f34a5255 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:37.804220 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a733a90-3637-4b7c-9240-9356f34a5255 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 02:15:37.813791 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-7a733a90-3637-4b7c-9240-9356f34a5255 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 02:15:37.813791 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:37.813791 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:37.813791 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:37.813791 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:37.813791 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:37.813791 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:37.813791 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:37.813791 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:37.813791 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:37.813791 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:37.813791 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:37.813791 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 02:15:37.813791 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 02:15:37.813791 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 02:15:37.813791 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 02:15:37.814868 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 02:15:37.814868 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 02:15:37.814868 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 02:15:37.814868 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 02:15:37.814868 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:37.814868 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:37.814868 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:37.814868 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:37.814868 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:37.814868 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:37.814868 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:37.814868 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:37.814868 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:37.814868 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:37.814868 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:37.814868 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:37.815793 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:37.815793 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:37.815793 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:37.815793 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:37.815793 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 02:15:37.815793 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 02:15:37.815793 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 02:15:37.815793 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 02:15:37.815793 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 02:15:37.815793 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 02:15:37.815793 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 02:15:37.815793 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:15:37.815793 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 337/667] 10.222.0.66 () {56 vars in 1102 bytes} [Mon Jul 18 02:15:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:37.816523 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a733a90-3637-4b7c-9240-9356f34a5255 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:37.817198 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fb20ea-6d5e-4568-8e9a-9deee0cf669c None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:37.864174 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecad2f8-0bbd-4055-bc72-8191ebce85b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:37.864835 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecad2f8-0bbd-4055-bc72-8191ebce85b1 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:37.865117 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecad2f8-0bbd-4055-bc72-8191ebce85b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:37.865392 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecad2f8-0bbd-4055-bc72-8191ebce85b1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:37.940579 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a733a90-3637-4b7c-9240-9356f34a5255 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:37.951503 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:37.958732 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecad2f8-0bbd-4055-bc72-8191ebce85b1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:37.960227 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecad2f8-0bbd-4055-bc72-8191ebce85b1 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:37.988622 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0249291b-414c-44a1-a34a-8e91233cf02a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:37.989716 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0249291b-414c-44a1-a34a-8e91233cf02a None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:37.990004 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0249291b-414c-44a1-a34a-8e91233cf02a None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:37.990452 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0249291b-414c-44a1-a34a-8e91233cf02a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:37.992958 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:38.072711 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 331/668] 10.222.0.66 () {58 vars in 1282 bytes} [Mon Jul 18 02:15:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 02:15:38.073887 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecad2f8-0bbd-4055-bc72-8191ebce85b1 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:38.083561 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0249291b-414c-44a1-a34a-8e91233cf02a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:38.085535 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0249291b-414c-44a1-a34a-8e91233cf02a None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:38.213270 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecad2f8-0bbd-4055-bc72-8191ebce85b1 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:38.230110 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 338/669] 10.222.0.66 () {58 vars in 1282 bytes} [Mon Jul 18 02:15:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 02:15:38.230110 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0249291b-414c-44a1-a34a-8e91233cf02a None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:38.240532 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:38.271362 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:38.287626 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea23b53-28fd-4853-960c-c6bf263a2785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:38.288318 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea23b53-28fd-4853-960c-c6bf263a2785 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:38.288699 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea23b53-28fd-4853-960c-c6bf263a2785 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:38.288980 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea23b53-28fd-4853-960c-c6bf263a2785 None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:38.289742 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea23b53-28fd-4853-960c-c6bf263a2785 None admin] RBAC: Adding query filter params (name=service) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:38.290052 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea23b53-28fd-4853-960c-c6bf263a2785 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:38.299708 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea23b53-28fd-4853-960c-c6bf263a2785 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:38.316189 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 332/670] 10.222.0.66 () {56 vars in 1103 bytes} [Mon Jul 18 02:15:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:38.317118 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea23b53-28fd-4853-960c-c6bf263a2785 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:38.355138 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0249291b-414c-44a1-a34a-8e91233cf02a None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:38.401088 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608ca908-f45d-4c2d-809b-bd5d734f2c85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:38.402444 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608ca908-f45d-4c2d-809b-bd5d734f2c85 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:38.403244 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608ca908-f45d-4c2d-809b-bd5d734f2c85 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:38.404190 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608ca908-f45d-4c2d-809b-bd5d734f2c85 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:38.416304 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608ca908-f45d-4c2d-809b-bd5d734f2c85 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:38.418004 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608ca908-f45d-4c2d-809b-bd5d734f2c85 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:38.440417 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-608ca908-f45d-4c2d-809b-bd5d734f2c85 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:15:38.440417 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:38.440417 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:38.440417 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:38.440417 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:38.440417 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:38.440417 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:38.440417 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:38.440417 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:38.440417 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:38.440417 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:38.440417 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:38.440417 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 02:15:38.440417 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 02:15:38.440417 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 02:15:38.441460 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 02:15:38.441460 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:38.441460 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:38.441460 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:38.441460 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:38.441460 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:38.441460 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:38.441460 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:38.441460 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:38.441460 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:38.441460 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:38.441460 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:38.441460 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:38.441460 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:38.441460 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:38.441460 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:38.441460 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:38.442741 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 02:15:38.442741 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 02:15:38.442741 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:15:38.442741 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:15:38.445013 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 339/671] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 18 02:15:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:38.445995 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608ca908-f45d-4c2d-809b-bd5d734f2c85 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:38.515367 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea23b53-28fd-4853-960c-c6bf263a2785 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:38.518673 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:38.565352 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608ca908-f45d-4c2d-809b-bd5d734f2c85 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:38.589971 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b881ac-6801-457b-b310-e95a494dbd97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:38.593310 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b881ac-6801-457b-b310-e95a494dbd97 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:38.593600 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b881ac-6801-457b-b310-e95a494dbd97 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:38.593880 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b881ac-6801-457b-b310-e95a494dbd97 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:38.618177 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b881ac-6801-457b-b310-e95a494dbd97 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:38.619266 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb017f33-60ae-44ab-b145-c495519e1adc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:38.619692 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b881ac-6801-457b-b310-e95a494dbd97 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:38.620718 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb017f33-60ae-44ab-b145-c495519e1adc None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:38.621229 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb017f33-60ae-44ab-b145-c495519e1adc None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:38.622591 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb017f33-60ae-44ab-b145-c495519e1adc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:38.634828 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb017f33-60ae-44ab-b145-c495519e1adc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:38.636882 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb017f33-60ae-44ab-b145-c495519e1adc None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:38.655988 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-cb017f33-60ae-44ab-b145-c495519e1adc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:15:38.655988 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:38.655988 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:38.655988 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:38.655988 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:38.655988 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:38.655988 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:38.655988 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:38.655988 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:38.655988 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:38.655988 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:38.655988 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:38.655988 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 02:15:38.655988 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 02:15:38.655988 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 02:15:38.656957 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 02:15:38.656957 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:38.656957 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:38.656957 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:38.656957 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:38.656957 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:38.656957 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:38.656957 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:38.656957 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:38.656957 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:38.656957 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:38.656957 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:38.656957 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:38.656957 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:38.656957 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:38.656957 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:38.656957 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:38.657939 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 02:15:38.657939 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 02:15:38.657939 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:15:38.657939 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:15:38.664554 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-a7b881ac-6801-457b-b310-e95a494dbd97 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:15:38.664554 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:38.664554 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:38.664554 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:38.664554 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:38.664554 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:38.664554 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:38.664554 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:38.664554 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:38.664554 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:38.664554 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:38.664554 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:38.664554 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 02:15:38.664554 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 02:15:38.664554 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 02:15:38.665479 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 02:15:38.665479 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:38.665479 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:38.665479 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:38.665479 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:38.665479 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:38.665479 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:38.665479 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:38.665479 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:38.665479 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:38.665479 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:38.665479 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:38.665479 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:38.665479 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:38.665479 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:38.665479 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:38.665479 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:38.666401 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 02:15:38.666401 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 02:15:38.666401 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:15:38.666401 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:15:38.667095 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 340/672] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 18 02:15:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:38.668730 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb017f33-60ae-44ab-b145-c495519e1adc None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:38.672320 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 333/673] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 18 02:15:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 355 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:38.673125 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b881ac-6801-457b-b310-e95a494dbd97 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:38.789961 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb017f33-60ae-44ab-b145-c495519e1adc None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:38.795157 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:38.809522 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:38.833375 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179aaef7-724c-4046-aac1-4e789df6ad4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:38.834553 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179aaef7-724c-4046-aac1-4e789df6ad4b None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:38.835060 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179aaef7-724c-4046-aac1-4e789df6ad4b None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:38.835665 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179aaef7-724c-4046-aac1-4e789df6ad4b None admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:38.836611 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179aaef7-724c-4046-aac1-4e789df6ad4b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:38.837280 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179aaef7-724c-4046-aac1-4e789df6ad4b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:38.838898 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179aaef7-724c-4046-aac1-4e789df6ad4b None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:38.855652 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 341/674] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 18 02:15:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:38.857657 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179aaef7-724c-4046-aac1-4e789df6ad4b None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:38.899462 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b881ac-6801-457b-b310-e95a494dbd97 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:38.935955 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc33c900-a457-468b-a73d-b052907d316a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:38.936631 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc33c900-a457-468b-a73d-b052907d316a None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:38.936973 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc33c900-a457-468b-a73d-b052907d316a None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:38.937246 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc33c900-a457-468b-a73d-b052907d316a None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:38.938010 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc33c900-a457-468b-a73d-b052907d316a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:38.938349 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc33c900-a457-468b-a73d-b052907d316a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:38.939841 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc33c900-a457-468b-a73d-b052907d316a None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:38.958423 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 334/675] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 18 02:15:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:38.959433 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc33c900-a457-468b-a73d-b052907d316a None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:38.989625 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179aaef7-724c-4046-aac1-4e789df6ad4b None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:39.028648 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f4c0d0-49ca-42a6-be0f-0efb5e868a36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:39.030381 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f4c0d0-49ca-42a6-be0f-0efb5e868a36 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:39.031028 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f4c0d0-49ca-42a6-be0f-0efb5e868a36 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:39.031619 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f4c0d0-49ca-42a6-be0f-0efb5e868a36 None admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:39.032826 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f4c0d0-49ca-42a6-be0f-0efb5e868a36 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:39.033705 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f4c0d0-49ca-42a6-be0f-0efb5e868a36 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:39.035859 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f4c0d0-49ca-42a6-be0f-0efb5e868a36 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:39.053321 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 342/676] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 18 02:15:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:39.055167 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f4c0d0-49ca-42a6-be0f-0efb5e868a36 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:39.078871 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc33c900-a457-468b-a73d-b052907d316a None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:39.080678 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:39.095807 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:39.117484 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dcd6d5-19a0-4abd-85ee-cef922b02d7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:39.118410 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dcd6d5-19a0-4abd-85ee-cef922b02d7e None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:39.118696 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dcd6d5-19a0-4abd-85ee-cef922b02d7e None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:39.118973 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dcd6d5-19a0-4abd-85ee-cef922b02d7e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:39.125742 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dcd6d5-19a0-4abd-85ee-cef922b02d7e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:39.127299 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dcd6d5-19a0-4abd-85ee-cef922b02d7e None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:39.142505 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-99dcd6d5-19a0-4abd-85ee-cef922b02d7e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 02:15:39.142505 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:39.142505 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:39.142505 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:39.142505 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:39.142505 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:39.142505 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:39.142505 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:39.142505 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:39.142505 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:39.142505 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:39.142505 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:39.142505 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 02:15:39.142505 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 02:15:39.142505 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 02:15:39.143437 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 02:15:39.143437 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:39.143437 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:39.143437 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:39.143437 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:39.143437 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:39.143437 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:39.143437 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:39.143437 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:39.143437 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:39.143437 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:39.143437 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:39.143437 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:39.143437 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:39.143437 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:39.143437 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:39.143437 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:39.144399 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 02:15:39.144399 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 02:15:39.144399 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 02:15:39.144399 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 02:15:39.144399 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 02:15:39.144399 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 02:15:39.144399 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 02:15:39.144399 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:15:39.144399 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 335/677] 10.222.0.66 () {56 vars in 1111 bytes} [Mon Jul 18 02:15:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:39.144925 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dcd6d5-19a0-4abd-85ee-cef922b02d7e None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:39.172369 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f4c0d0-49ca-42a6-be0f-0efb5e868a36 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:39.212743 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba73f226-ca34-448d-92c4-5a5baf26940d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:39.213804 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba73f226-ca34-448d-92c4-5a5baf26940d None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:39.214112 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba73f226-ca34-448d-92c4-5a5baf26940d None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:39.214562 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba73f226-ca34-448d-92c4-5a5baf26940d None admin] PATH_INFO: `/v3/users/swift` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:39.224780 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba73f226-ca34-448d-92c4-5a5baf26940d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:39.226258 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba73f226-ca34-448d-92c4-5a5baf26940d None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:39.243410 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-ba73f226-ca34-448d-92c4-5a5baf26940d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 18 02:15:39.243410 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:39.243410 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:39.243410 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:39.243410 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:39.243410 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:39.243410 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:39.243410 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:39.243410 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:39.243410 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:39.243410 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:39.243410 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:39.243410 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 02:15:39.243410 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 02:15:39.243410 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 02:15:39.243410 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 02:15:39.244382 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:39.244382 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:39.244382 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 02:15:39.244382 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 02:15:39.244382 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 02:15:39.244382 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 02:15:39.244382 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:39.244382 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:39.244382 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:39.244382 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:39.244382 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:39.244382 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:39.244382 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:39.244382 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:39.244382 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:39.244382 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:39.244382 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:39.244382 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:39.245369 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:39.245369 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:39.245369 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 02:15:39.245369 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 02:15:39.245369 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 02:15:39.245369 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 02:15:39.245369 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 02:15:39.245369 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 02:15:39.245369 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 18 02:15:39.245369 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:15:39.245369 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 343/678] 10.222.0.66 () {56 vars in 1096 bytes} [Mon Jul 18 02:15:39 2022] GET /identity/v3/users/swift => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:39.246005 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9956b69b-56fd-48ae-836c-acb4a9104de5 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:39.247124 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9956b69b-56fd-48ae-836c-acb4a9104de5 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:39.247124 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9956b69b-56fd-48ae-836c-acb4a9104de5 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:39.248543 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 344/679] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:15:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:15:39.249125 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9956b69b-56fd-48ae-836c-acb4a9104de5 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:39.261424 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dcd6d5-19a0-4abd-85ee-cef922b02d7e None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:39.296011 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59151e31-4288-4e97-9401-b0a7632f60d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:39.296788 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59151e31-4288-4e97-9401-b0a7632f60d3 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:39.297331 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59151e31-4288-4e97-9401-b0a7632f60d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:39.297608 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59151e31-4288-4e97-9401-b0a7632f60d3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:39.304521 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59151e31-4288-4e97-9401-b0a7632f60d3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:39.305918 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59151e31-4288-4e97-9401-b0a7632f60d3 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:39.320883 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-59151e31-4288-4e97-9401-b0a7632f60d3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 02:15:39.320883 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:39.320883 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:39.320883 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:39.320883 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:39.320883 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:39.320883 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:39.320883 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:39.320883 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:39.320883 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:39.320883 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:39.320883 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:39.320883 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 02:15:39.320883 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 02:15:39.320883 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 02:15:39.321799 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 02:15:39.321799 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:39.321799 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:39.321799 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:39.321799 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:39.321799 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:39.321799 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:39.321799 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:39.321799 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:39.321799 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:39.321799 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:39.321799 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:39.321799 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:39.321799 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:39.321799 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:39.321799 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:39.321799 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:39.322735 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 02:15:39.322735 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 02:15:39.322735 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 02:15:39.322735 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 02:15:39.322735 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 02:15:39.322735 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 02:15:39.322735 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 02:15:39.322735 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:15:39.322735 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 336/680] 10.222.0.66 () {56 vars in 1111 bytes} [Mon Jul 18 02:15:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:39.323664 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59151e31-4288-4e97-9401-b0a7632f60d3 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:39.350587 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:39.356641 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:39.366145 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9956b69b-56fd-48ae-836c-acb4a9104de5 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:39.404090 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b9f24d-c17d-4f15-adf7-fba3f24b8732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:39.405175 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b9f24d-c17d-4f15-adf7-fba3f24b8732 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:39.405466 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b9f24d-c17d-4f15-adf7-fba3f24b8732 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:39.405745 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b9f24d-c17d-4f15-adf7-fba3f24b8732 None admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:39.406914 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b9f24d-c17d-4f15-adf7-fba3f24b8732 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:39.407232 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b9f24d-c17d-4f15-adf7-fba3f24b8732 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:39.409374 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b9f24d-c17d-4f15-adf7-fba3f24b8732 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:39.423392 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 345/681] 10.222.0.66 () {56 vars in 1148 bytes} [Mon Jul 18 02:15:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:39.424659 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9105a958-10ea-467d-b52c-1d446f5fec13 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:39.424951 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9105a958-10ea-467d-b52c-1d446f5fec13 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:39.425229 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9105a958-10ea-467d-b52c-1d446f5fec13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:39.435379 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59151e31-4288-4e97-9401-b0a7632f60d3 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:39.443575 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-9105a958-10ea-467d-b52c-1d446f5fec13 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:15:39.444874 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-9105a958-10ea-467d-b52c-1d446f5fec13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:15:39.472132 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85cdf86-fd93-4dbc-9279-6f586ec2392a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:39.472801 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85cdf86-fd93-4dbc-9279-6f586ec2392a None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:39.473082 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85cdf86-fd93-4dbc-9279-6f586ec2392a None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:39.473818 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85cdf86-fd93-4dbc-9279-6f586ec2392a None admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:39.475254 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85cdf86-fd93-4dbc-9279-6f586ec2392a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:39.475569 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85cdf86-fd93-4dbc-9279-6f586ec2392a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:39.477212 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85cdf86-fd93-4dbc-9279-6f586ec2392a None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:39.501718 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 337/682] 10.222.0.66 () {56 vars in 1135 bytes} [Mon Jul 18 02:15:39 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:39.501718 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85cdf86-fd93-4dbc-9279-6f586ec2392a None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:39.520764 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 346/683] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:15:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 02:15:39.526309 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9105a958-10ea-467d-b52c-1d446f5fec13 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:39.595358 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:39.619793 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:39.621076 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85cdf86-fd93-4dbc-9279-6f586ec2392a None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:39.637731 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9105a958-10ea-467d-b52c-1d446f5fec13 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:39.659154 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9645939c-d9a1-4ee9-b18c-eeae69005ce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:39.660251 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9645939c-d9a1-4ee9-b18c-eeae69005ce1 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:39.660542 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9645939c-d9a1-4ee9-b18c-eeae69005ce1 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:39.660899 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9645939c-d9a1-4ee9-b18c-eeae69005ce1 None admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:39.661607 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9645939c-d9a1-4ee9-b18c-eeae69005ce1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:39.661920 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9645939c-d9a1-4ee9-b18c-eeae69005ce1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:39.663501 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9645939c-d9a1-4ee9-b18c-eeae69005ce1 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:39.674678 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58150a13-d462-4355-b497-09efed7ea0d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:39.675351 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58150a13-d462-4355-b497-09efed7ea0d5 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:39.675609 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58150a13-d462-4355-b497-09efed7ea0d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:39.675884 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58150a13-d462-4355-b497-09efed7ea0d5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:39.676620 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 338/684] 10.222.0.66 () {56 vars in 1148 bytes} [Mon Jul 18 02:15:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:39.676744 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58150a13-d462-4355-b497-09efed7ea0d5 None admin] RBAC: Adding query filter params (role.id=eea94daf3312421cb12aa8abfa73160e, scope.project.id=76f7c0ae36ae49b09dcae4c3e9a875f4, user.id=cd49388facdf446885e55b45ef91f01e) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:39.676997 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58150a13-d462-4355-b497-09efed7ea0d5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:39.678459 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58150a13-d462-4355-b497-09efed7ea0d5 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:39.681997 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9645939c-d9a1-4ee9-b18c-eeae69005ce1 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:39.684772 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 347/685] 10.222.0.66 () {56 vars in 1374 bytes} [Mon Jul 18 02:15:39 2022] GET /identity/v3/role_assignments?user.id=cd49388facdf446885e55b45ef91f01e&scope.project.id=76f7c0ae36ae49b09dcae4c3e9a875f4&role.id=eea94daf3312421cb12aa8abfa73160e => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:39.685481 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58150a13-d462-4355-b497-09efed7ea0d5 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:39.814272 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9645939c-d9a1-4ee9-b18c-eeae69005ce1 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:39.832563 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58150a13-d462-4355-b497-09efed7ea0d5 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:39.853226 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e030bc-5577-4610-a414-783a4d7a4091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:39.853816 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e030bc-5577-4610-a414-783a4d7a4091 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:39.854198 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e030bc-5577-4610-a414-783a4d7a4091 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:39.854772 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e030bc-5577-4610-a414-783a4d7a4091 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:39.871099 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:39.878799 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347db649-9cc0-4a31-bcd0-5fbb140c57fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:39.879828 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347db649-9cc0-4a31-bcd0-5fbb140c57fe None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:39.880270 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347db649-9cc0-4a31-bcd0-5fbb140c57fe None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:39.880667 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347db649-9cc0-4a31-bcd0-5fbb140c57fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:39.892781 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347db649-9cc0-4a31-bcd0-5fbb140c57fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:39.894311 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347db649-9cc0-4a31-bcd0-5fbb140c57fe None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:39.903485 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:39.913409 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-347db649-9cc0-4a31-bcd0-5fbb140c57fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:15:39.913409 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:39.913409 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:39.913409 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:39.913409 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:39.913409 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:39.913409 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:39.913409 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:39.913409 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:39.913409 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:39.913409 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:39.913409 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:39.913409 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 02:15:39.913409 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 02:15:39.913409 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 02:15:39.914400 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 02:15:39.914400 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:39.914400 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:39.914400 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:39.914400 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:39.914400 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:39.914400 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:39.914400 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:39.914400 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:39.914400 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:39.914400 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:39.914400 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:39.914400 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:39.914400 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:39.914400 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:39.914400 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:39.914400 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:39.915625 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 02:15:39.915625 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 02:15:39.915625 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:15:39.915625 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:15:39.915625 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 348/686] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 18 02:15:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:39.915905 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347db649-9cc0-4a31-bcd0-5fbb140c57fe None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:39.940005 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e030bc-5577-4610-a414-783a4d7a4091 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:39.941651 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e030bc-5577-4610-a414-783a4d7a4091 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:40.026315 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347db649-9cc0-4a31-bcd0-5fbb140c57fe None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:40.052935 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 339/687] 10.222.0.66 () {58 vars in 1282 bytes} [Mon Jul 18 02:15:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 02:15:40.052935 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6349606f-0ff4-46ac-98d6-676a4dadf08e None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:40.052935 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6349606f-0ff4-46ac-98d6-676a4dadf08e None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:40.052935 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6349606f-0ff4-46ac-98d6-676a4dadf08e None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:40.053352 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 340/688] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:15:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:15:40.054178 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6349606f-0ff4-46ac-98d6-676a4dadf08e None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:40.067743 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a436084d-0fed-43af-b206-fd4080d1a18e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:40.068404 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a436084d-0fed-43af-b206-fd4080d1a18e None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:40.068688 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a436084d-0fed-43af-b206-fd4080d1a18e None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:40.068970 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a436084d-0fed-43af-b206-fd4080d1a18e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:40.069800 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a436084d-0fed-43af-b206-fd4080d1a18e None admin] RBAC: Adding query filter params (role.id=eea94daf3312421cb12aa8abfa73160e, scope.project.id=76f7c0ae36ae49b09dcae4c3e9a875f4, user.id=eaf19a259c2741cea375a38763e33cdc) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:40.070143 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a436084d-0fed-43af-b206-fd4080d1a18e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:40.071594 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a436084d-0fed-43af-b206-fd4080d1a18e None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:40.078209 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 349/689] 10.222.0.66 () {56 vars in 1374 bytes} [Mon Jul 18 02:15:39 2022] GET /identity/v3/role_assignments?user.id=eaf19a259c2741cea375a38763e33cdc&scope.project.id=76f7c0ae36ae49b09dcae4c3e9a875f4&role.id=eea94daf3312421cb12aa8abfa73160e => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:40.079008 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a436084d-0fed-43af-b206-fd4080d1a18e None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:40.121024 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:40.151671 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:40.204487 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6349606f-0ff4-46ac-98d6-676a4dadf08e None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:40.208419 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a436084d-0fed-43af-b206-fd4080d1a18e None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:40.245342 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43385a5f-2236-4a6b-972f-8780c886247f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:40.247101 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43385a5f-2236-4a6b-972f-8780c886247f None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:40.247931 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43385a5f-2236-4a6b-972f-8780c886247f None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:40.247931 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43385a5f-2236-4a6b-972f-8780c886247f None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:40.248527 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43385a5f-2236-4a6b-972f-8780c886247f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:40.248833 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43385a5f-2236-4a6b-972f-8780c886247f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:40.250275 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43385a5f-2236-4a6b-972f-8780c886247f None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:40.263139 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b06af2-2932-45f1-8f5b-dd9f51d41243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:40.264246 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b06af2-2932-45f1-8f5b-dd9f51d41243 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:40.265516 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b06af2-2932-45f1-8f5b-dd9f51d41243 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:40.265982 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b06af2-2932-45f1-8f5b-dd9f51d41243 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:40.268796 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 341/690] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 18 02:15:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:40.270629 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e32b41-5efa-4f34-9498-f16855502dda None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:40.270923 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e32b41-5efa-4f34-9498-f16855502dda None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:40.271204 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e32b41-5efa-4f34-9498-f16855502dda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:40.280003 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b06af2-2932-45f1-8f5b-dd9f51d41243 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:40.281473 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b06af2-2932-45f1-8f5b-dd9f51d41243 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:40.290926 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-02e32b41-5efa-4f34-9498-f16855502dda None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:15:40.292519 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-02e32b41-5efa-4f34-9498-f16855502dda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:15:40.303753 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-e9b06af2-2932-45f1-8f5b-dd9f51d41243 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:15:40.303753 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:40.303753 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:40.303753 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:40.303753 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:40.303753 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:40.303753 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:40.303753 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:40.303753 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:40.303753 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:40.303753 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:40.303753 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:40.303753 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 02:15:40.303753 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 02:15:40.303753 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 02:15:40.304712 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 02:15:40.304712 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:40.304712 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:40.304712 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:40.304712 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:40.304712 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:40.304712 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:40.304712 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:40.304712 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:40.304712 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:40.304712 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:40.304712 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:40.304712 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:40.304712 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:40.304712 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:40.304712 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:40.304712 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:40.305674 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 02:15:40.305674 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 02:15:40.305674 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:15:40.305674 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:15:40.305674 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 350/691] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 18 02:15:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:40.305986 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b06af2-2932-45f1-8f5b-dd9f51d41243 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:40.393155 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 342/692] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:15:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 02:15:40.393975 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e32b41-5efa-4f34-9498-f16855502dda None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:40.426461 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:40.433909 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b06af2-2932-45f1-8f5b-dd9f51d41243 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:40.453458 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:40.483566 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a379a2-5cf2-4493-ace0-63f85a8d912e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:40.484785 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a379a2-5cf2-4493-ace0-63f85a8d912e None admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:40.485220 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a379a2-5cf2-4493-ace0-63f85a8d912e None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:40.486061 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a379a2-5cf2-4493-ace0-63f85a8d912e None admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:40.487288 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a379a2-5cf2-4493-ace0-63f85a8d912e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:40.489605 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a379a2-5cf2-4493-ace0-63f85a8d912e None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:40.505742 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e32b41-5efa-4f34-9498-f16855502dda None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:40.541851 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc34d19-11cb-4008-8653-463a8bdb9226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:40.542557 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc34d19-11cb-4008-8653-463a8bdb9226 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:40.542840 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc34d19-11cb-4008-8653-463a8bdb9226 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:40.543116 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc34d19-11cb-4008-8653-463a8bdb9226 None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:40.544719 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc34d19-11cb-4008-8653-463a8bdb9226 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:40.545031 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc34d19-11cb-4008-8653-463a8bdb9226 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:40.546440 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc34d19-11cb-4008-8653-463a8bdb9226 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:40.565994 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 343/693] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 18 02:15:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:40.565994 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc34d19-11cb-4008-8653-463a8bdb9226 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:40.572064 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 351/694] 10.222.0.66 () {60 vars in 1132 bytes} [Mon Jul 18 02:15:40 2022] POST /identity/v3/users => generated 254 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:15:40.574575 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a379a2-5cf2-4493-ace0-63f85a8d912e None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:40.748716 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:40.764513 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a379a2-5cf2-4493-ace0-63f85a8d912e None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:40.767854 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc34d19-11cb-4008-8653-463a8bdb9226 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:40.823125 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c19974e-ef4b-4afd-84f6-de3010fc7510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:40.823933 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c19974e-ef4b-4afd-84f6-de3010fc7510 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:40.824217 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c19974e-ef4b-4afd-84f6-de3010fc7510 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:40.824524 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c19974e-ef4b-4afd-84f6-de3010fc7510 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:40.830033 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e285a2c-ec22-4368-83c9-ae7e231fa36f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:40.830909 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e285a2c-ec22-4368-83c9-ae7e231fa36f None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:40.831777 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e285a2c-ec22-4368-83c9-ae7e231fa36f None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:40.831777 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e285a2c-ec22-4368-83c9-ae7e231fa36f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:40.843920 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e285a2c-ec22-4368-83c9-ae7e231fa36f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:40.845722 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e285a2c-ec22-4368-83c9-ae7e231fa36f None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:40.846834 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c19974e-ef4b-4afd-84f6-de3010fc7510 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 02:15:40.854767 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-7c19974e-ef4b-4afd-84f6-de3010fc7510 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 02:15:40.854767 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:40.854767 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:40.854767 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:40.854767 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:40.854767 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:40.854767 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:40.854767 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:40.854767 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:40.854767 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:40.854767 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:40.854767 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:40.854767 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 02:15:40.854767 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 02:15:40.854767 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 02:15:40.854767 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 02:15:40.856605 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 02:15:40.856605 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 02:15:40.856605 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 02:15:40.856605 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 02:15:40.856605 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:40.856605 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:40.856605 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:40.856605 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:40.856605 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:40.856605 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:40.856605 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:40.856605 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:40.856605 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:40.856605 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:40.856605 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:40.856605 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:40.857792 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:40.857792 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:40.857792 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:40.857792 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:40.857792 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 02:15:40.857792 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 02:15:40.857792 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 02:15:40.857792 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 02:15:40.857792 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 02:15:40.857792 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 02:15:40.857792 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 02:15:40.857792 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:15:40.858753 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 344/695] 10.222.0.66 () {56 vars in 1096 bytes} [Mon Jul 18 02:15:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:40.867069 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-7e285a2c-ec22-4368-83c9-ae7e231fa36f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 02:15:40.867069 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:40.867069 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:40.867069 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:40.867069 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:40.867069 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:40.867069 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:40.867069 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:40.867069 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:40.867069 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:40.867069 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:40.867069 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:40.867069 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 02:15:40.867069 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 02:15:40.867069 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 02:15:40.867985 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 02:15:40.867985 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:40.867985 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:40.867985 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:40.867985 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:40.867985 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:40.867985 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:40.867985 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:40.867985 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:40.867985 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:40.867985 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:40.867985 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:40.867985 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:40.867985 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:40.867985 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:40.867985 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:40.867985 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:40.868901 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 02:15:40.868901 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 02:15:40.868901 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 02:15:40.868901 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 02:15:40.868901 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 02:15:40.868901 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 02:15:40.868901 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 02:15:40.868901 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:15:40.869334 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c19974e-ef4b-4afd-84f6-de3010fc7510 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:40.871340 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 352/696] 10.222.0.66 () {56 vars in 1111 bytes} [Mon Jul 18 02:15:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:40.886227 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e285a2c-ec22-4368-83c9-ae7e231fa36f None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:40.924942 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:40.994003 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c19974e-ef4b-4afd-84f6-de3010fc7510 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:41.013282 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e285a2c-ec22-4368-83c9-ae7e231fa36f None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:41.035376 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdec306-0210-4904-b6dc-5eb0148f148b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:41.036159 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdec306-0210-4904-b6dc-5eb0148f148b None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:41.036489 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdec306-0210-4904-b6dc-5eb0148f148b None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:41.036765 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdec306-0210-4904-b6dc-5eb0148f148b None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:41.037520 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdec306-0210-4904-b6dc-5eb0148f148b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:41.037825 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdec306-0210-4904-b6dc-5eb0148f148b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:41.039258 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdec306-0210-4904-b6dc-5eb0148f148b None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:41.049586 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 345/697] 10.222.0.66 () {56 vars in 1099 bytes} [Mon Jul 18 02:15:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:41.054959 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdec306-0210-4904-b6dc-5eb0148f148b None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:41.071724 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:41.104243 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbefa76-0535-4420-a6fd-b6354929e57e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:41.104927 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbefa76-0535-4420-a6fd-b6354929e57e None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:41.105207 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbefa76-0535-4420-a6fd-b6354929e57e None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:41.105480 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbefa76-0535-4420-a6fd-b6354929e57e None admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:41.106536 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbefa76-0535-4420-a6fd-b6354929e57e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:41.106853 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbefa76-0535-4420-a6fd-b6354929e57e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:41.108413 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbefa76-0535-4420-a6fd-b6354929e57e None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:41.128558 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 353/698] 10.222.0.66 () {56 vars in 1148 bytes} [Mon Jul 18 02:15:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 02:15:41.134679 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbefa76-0535-4420-a6fd-b6354929e57e None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:41.164242 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdec306-0210-4904-b6dc-5eb0148f148b None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:41.202622 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705e1689-4870-4e0c-9e85-b0e73539c11b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:41.203533 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705e1689-4870-4e0c-9e85-b0e73539c11b None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:41.203808 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705e1689-4870-4e0c-9e85-b0e73539c11b None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:41.204091 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705e1689-4870-4e0c-9e85-b0e73539c11b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:41.215157 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705e1689-4870-4e0c-9e85-b0e73539c11b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:41.216994 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705e1689-4870-4e0c-9e85-b0e73539c11b None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:41.237011 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-705e1689-4870-4e0c-9e85-b0e73539c11b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:15:41.237011 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:41.237011 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:41.237011 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:41.237011 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:41.237011 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:41.237011 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:41.237011 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:41.237011 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:41.237011 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:41.237011 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:41.237011 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:41.237011 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 02:15:41.237011 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 02:15:41.237011 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 02:15:41.237949 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 02:15:41.237949 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:41.237949 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:41.237949 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:41.237949 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:41.237949 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:41.237949 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:41.237949 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:41.237949 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:41.237949 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:41.237949 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:41.237949 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:41.237949 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:41.237949 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:41.237949 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:41.237949 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:41.237949 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:41.238916 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 02:15:41.238916 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 02:15:41.238916 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:15:41.238916 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:15:41.238916 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 346/699] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 18 02:15:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:41.244202 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705e1689-4870-4e0c-9e85-b0e73539c11b None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:41.338197 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:41.350886 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:41.356680 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbefa76-0535-4420-a6fd-b6354929e57e None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:41.432011 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705e1689-4870-4e0c-9e85-b0e73539c11b None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:41.435610 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8650655-8434-4c9b-98c8-149980f3c353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:41.436280 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8650655-8434-4c9b-98c8-149980f3c353 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:41.436568 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8650655-8434-4c9b-98c8-149980f3c353 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:41.436846 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8650655-8434-4c9b-98c8-149980f3c353 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:41.437679 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8650655-8434-4c9b-98c8-149980f3c353 None admin] RBAC: Adding query filter params (role.id=eea94daf3312421cb12aa8abfa73160e, scope.project.id=76f7c0ae36ae49b09dcae4c3e9a875f4, user.id=b4300622cbd347dd8f5227bb960e98d0) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:41.437992 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8650655-8434-4c9b-98c8-149980f3c353 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:41.439466 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8650655-8434-4c9b-98c8-149980f3c353 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:41.451996 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 354/700] 10.222.0.66 () {56 vars in 1374 bytes} [Mon Jul 18 02:15:41 2022] GET /identity/v3/role_assignments?user.id=b4300622cbd347dd8f5227bb960e98d0&scope.project.id=76f7c0ae36ae49b09dcae4c3e9a875f4&role.id=eea94daf3312421cb12aa8abfa73160e => generated 608 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 02:15:41.482133 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5d77aa-0610-4888-ad1b-94b01ea2b800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:41.482916 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5d77aa-0610-4888-ad1b-94b01ea2b800 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:41.483276 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5d77aa-0610-4888-ad1b-94b01ea2b800 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:41.483628 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5d77aa-0610-4888-ad1b-94b01ea2b800 None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:41.484447 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5d77aa-0610-4888-ad1b-94b01ea2b800 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:41.484879 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5d77aa-0610-4888-ad1b-94b01ea2b800 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:41.486480 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5d77aa-0610-4888-ad1b-94b01ea2b800 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:41.501680 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 347/701] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 18 02:15:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:41.506965 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8650655-8434-4c9b-98c8-149980f3c353 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:41.623639 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8650655-8434-4c9b-98c8-149980f3c353 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:41.665254 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac58ea0-e464-4267-98cb-2cb7253b983a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:41.666054 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac58ea0-e464-4267-98cb-2cb7253b983a None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:41.666375 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac58ea0-e464-4267-98cb-2cb7253b983a None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:41.666653 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac58ea0-e464-4267-98cb-2cb7253b983a None admin] PATH_INFO: `/v3/users/nova` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:41.676213 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac58ea0-e464-4267-98cb-2cb7253b983a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:41.677745 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac58ea0-e464-4267-98cb-2cb7253b983a None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:41.692532 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-cac58ea0-e464-4267-98cb-2cb7253b983a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 18 02:15:41.692532 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:41.692532 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:41.692532 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:41.692532 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:41.692532 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:41.692532 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:41.692532 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:41.692532 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:41.692532 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:41.692532 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:41.692532 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:41.692532 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 02:15:41.692532 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 02:15:41.692532 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 02:15:41.692532 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 02:15:41.693545 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:41.693545 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:41.693545 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 02:15:41.693545 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 02:15:41.693545 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 02:15:41.693545 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 02:15:41.693545 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:41.693545 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:41.693545 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:41.693545 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:41.693545 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:41.693545 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:41.693545 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:41.693545 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:41.693545 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:41.693545 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:41.693545 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:41.693545 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:41.695121 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:41.695121 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:41.695121 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 02:15:41.695121 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 02:15:41.695121 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 02:15:41.695121 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 02:15:41.695121 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 02:15:41.695121 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 02:15:41.695121 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 18 02:15:41.695121 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:15:41.695121 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 355/702] 10.222.0.66 () {56 vars in 1093 bytes} [Mon Jul 18 02:15:41 2022] GET /identity/v3/users/nova => generated 82 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:41.699353 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5d77aa-0610-4888-ad1b-94b01ea2b800 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:41.818932 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5d77aa-0610-4888-ad1b-94b01ea2b800 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:41.842957 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:41.853378 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c79d29-34f5-4d14-8d03-bf8de3d492c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:41.859932 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c79d29-34f5-4d14-8d03-bf8de3d492c1 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:41.859932 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c79d29-34f5-4d14-8d03-bf8de3d492c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:41.859932 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c79d29-34f5-4d14-8d03-bf8de3d492c1 None admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:41.859932 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c79d29-34f5-4d14-8d03-bf8de3d492c1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:41.859932 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c79d29-34f5-4d14-8d03-bf8de3d492c1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:41.859932 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c79d29-34f5-4d14-8d03-bf8de3d492c1 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:41.878541 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 348/703] 10.222.0.66 () {56 vars in 1133 bytes} [Mon Jul 18 02:15:41 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:41.883576 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac58ea0-e464-4267-98cb-2cb7253b983a None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:41.901618 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:41.997116 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac58ea0-e464-4267-98cb-2cb7253b983a None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:42.032540 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17edc74-7252-4a06-a2bc-3cb5ae16a66a None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:42.033087 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17edc74-7252-4a06-a2bc-3cb5ae16a66a None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:42.033087 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17edc74-7252-4a06-a2bc-3cb5ae16a66a None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:42.034740 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 349/704] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:15:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 02:15:42.036538 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9520dff-7c22-4062-9083-5046a81c2706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:42.037198 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9520dff-7c22-4062-9083-5046a81c2706 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:42.037487 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9520dff-7c22-4062-9083-5046a81c2706 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:42.037769 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9520dff-7c22-4062-9083-5046a81c2706 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:42.043905 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48193be1-819f-495d-b197-9822cc2204c9 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:42.044348 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48193be1-819f-495d-b197-9822cc2204c9 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:42.044794 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48193be1-819f-495d-b197-9822cc2204c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:42.097949 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-48193be1-819f-495d-b197-9822cc2204c9 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:15:42.099183 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-48193be1-819f-495d-b197-9822cc2204c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:15:42.119382 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9520dff-7c22-4062-9083-5046a81c2706 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:42.121267 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9520dff-7c22-4062-9083-5046a81c2706 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:42.158075 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:42.222243 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 350/705] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:15:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 02:15:42.247980 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48193be1-819f-495d-b197-9822cc2204c9 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:42.320211 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 356/706] 10.222.0.66 () {58 vars in 1282 bytes} [Mon Jul 18 02:15:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 02:15:42.326042 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9520dff-7c22-4062-9083-5046a81c2706 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:42.420631 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48193be1-819f-495d-b197-9822cc2204c9 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:42.443271 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:42.484898 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2533ac08-734d-47a7-b475-acd2873151e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:42.485716 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2533ac08-734d-47a7-b475-acd2873151e2 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:42.485997 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2533ac08-734d-47a7-b475-acd2873151e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:42.486295 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2533ac08-734d-47a7-b475-acd2873151e2 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:42.486989 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2533ac08-734d-47a7-b475-acd2873151e2 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:42.488382 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2533ac08-734d-47a7-b475-acd2873151e2 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:42.508539 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-2533ac08-734d-47a7-b475-acd2873151e2 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 18 02:15:42.508539 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:42.508539 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 02:15:42.508539 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 02:15:42.508539 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 02:15:42.508539 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 02:15:42.508539 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 02:15:42.508539 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 02:15:42.508539 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 18 02:15:42.508539 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:15:42.508539 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 02:15:42.508539 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:15:42.508539 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:42.508539 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:42.508539 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:42.508539 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:42.509557 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:42.509557 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:42.509557 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:42.509557 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:42.509557 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:42.509557 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:42.509557 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:42.509557 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 02:15:42.509557 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 02:15:42.509557 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 02:15:42.509557 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 02:15:42.509557 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:42.509557 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:42.509557 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:42.509557 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:42.509557 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:42.510470 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:42.510470 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:42.510470 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:42.510470 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:42.510470 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:42.510470 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:42.510470 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:42.510470 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:42.510470 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:42.510470 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:42.510470 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:42.510470 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 02:15:42.510470 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 02:15:42.510470 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 18 02:15:42.510470 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:15:42.510470 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 351/707] 10.222.0.66 () {56 vars in 1129 bytes} [Mon Jul 18 02:15:42 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 02:15:42.516325 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9520dff-7c22-4062-9083-5046a81c2706 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:42.524145 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2533ac08-734d-47a7-b475-acd2873151e2 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:42.539348 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:42.576396 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c091bb6-2d55-4aaf-bf52-172687d5026e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:42.577193 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c091bb6-2d55-4aaf-bf52-172687d5026e None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:42.577471 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c091bb6-2d55-4aaf-bf52-172687d5026e None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:42.577740 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c091bb6-2d55-4aaf-bf52-172687d5026e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:42.592965 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c091bb6-2d55-4aaf-bf52-172687d5026e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:42.594876 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c091bb6-2d55-4aaf-bf52-172687d5026e None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:42.616634 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-0c091bb6-2d55-4aaf-bf52-172687d5026e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:15:42.616634 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:42.616634 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:42.616634 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:42.616634 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:42.616634 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:42.616634 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:42.616634 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:42.616634 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:42.616634 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:42.616634 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:42.616634 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:42.616634 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 02:15:42.616634 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 02:15:42.616634 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 02:15:42.617553 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 02:15:42.617553 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:42.617553 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:42.617553 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:42.617553 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:42.617553 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:42.617553 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:42.617553 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:42.617553 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:42.617553 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:42.617553 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:42.617553 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:42.617553 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:42.617553 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:42.617553 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:42.617553 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:42.617553 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:42.618535 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 02:15:42.618535 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 02:15:42.618535 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:15:42.618535 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:15:42.618535 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 357/708] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 18 02:15:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:42.619547 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdfa159-52e8-4f1d-9470-0c4ee9f23249 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:42.619828 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdfa159-52e8-4f1d-9470-0c4ee9f23249 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:42.620100 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdfa159-52e8-4f1d-9470-0c4ee9f23249 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:42.626402 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 358/709] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:15:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:15:42.626402 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdfa159-52e8-4f1d-9470-0c4ee9f23249 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:42.701288 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2533ac08-734d-47a7-b475-acd2873151e2 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:42.744180 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdfa159-52e8-4f1d-9470-0c4ee9f23249 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:42.749781 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6ad91f-5f52-4e07-8c44-da590203a9c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:42.750503 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6ad91f-5f52-4e07-8c44-da590203a9c2 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:42.750786 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6ad91f-5f52-4e07-8c44-da590203a9c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:42.751058 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6ad91f-5f52-4e07-8c44-da590203a9c2 None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:42.751765 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6ad91f-5f52-4e07-8c44-da590203a9c2 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:42.752068 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6ad91f-5f52-4e07-8c44-da590203a9c2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:42.753456 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6ad91f-5f52-4e07-8c44-da590203a9c2 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:42.760398 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 352/710] 10.222.0.66 () {56 vars in 1124 bytes} [Mon Jul 18 02:15:42 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:42.762599 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c664dd59-dc06-42ee-b46b-4ebf7f949dfc None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:42.763128 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c664dd59-dc06-42ee-b46b-4ebf7f949dfc None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:42.763413 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c664dd59-dc06-42ee-b46b-4ebf7f949dfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:42.784503 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-c664dd59-dc06-42ee-b46b-4ebf7f949dfc None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:15:42.785682 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-c664dd59-dc06-42ee-b46b-4ebf7f949dfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:15:42.786237 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f450338c-cdfb-4435-974c-c1500ec3f28b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:42.787193 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f450338c-cdfb-4435-974c-c1500ec3f28b None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:42.787693 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f450338c-cdfb-4435-974c-c1500ec3f28b None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:42.788166 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f450338c-cdfb-4435-974c-c1500ec3f28b None admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:42.789079 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f450338c-cdfb-4435-974c-c1500ec3f28b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:42.789582 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f450338c-cdfb-4435-974c-c1500ec3f28b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:42.791178 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f450338c-cdfb-4435-974c-c1500ec3f28b None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:42.804733 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 359/711] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 18 02:15:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 02:15:42.805989 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f450338c-cdfb-4435-974c-c1500ec3f28b None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:42.830735 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:42.839531 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:42.882145 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 353/712] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:15:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 02:15:42.882145 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c664dd59-dc06-42ee-b46b-4ebf7f949dfc None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:42.939345 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f450338c-cdfb-4435-974c-c1500ec3f28b None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:42.976821 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744567af-3953-4edd-97fc-773593091886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:42.977876 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744567af-3953-4edd-97fc-773593091886 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:42.978478 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744567af-3953-4edd-97fc-773593091886 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:42.979050 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744567af-3953-4edd-97fc-773593091886 None admin] PATH_INFO: `/v3/services` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:42.980028 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744567af-3953-4edd-97fc-773593091886 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:42.980571 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744567af-3953-4edd-97fc-773593091886 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:42.982392 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744567af-3953-4edd-97fc-773593091886 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:42.988461 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c664dd59-dc06-42ee-b46b-4ebf7f949dfc None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:42.989467 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 360/713] 10.222.0.66 () {56 vars in 1124 bytes} [Mon Jul 18 02:15:42 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:42.990710 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744567af-3953-4edd-97fc-773593091886 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:43.029424 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23b7579-162e-4c55-b251-883312fdcd84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:43.030277 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23b7579-162e-4c55-b251-883312fdcd84 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:43.030557 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23b7579-162e-4c55-b251-883312fdcd84 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:43.030829 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23b7579-162e-4c55-b251-883312fdcd84 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:43.038565 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23b7579-162e-4c55-b251-883312fdcd84 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:43.040035 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23b7579-162e-4c55-b251-883312fdcd84 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:43.056270 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-e23b7579-162e-4c55-b251-883312fdcd84 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 02:15:43.056270 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:43.056270 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:43.056270 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:43.056270 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:43.056270 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:43.056270 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:43.056270 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:43.056270 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:43.056270 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:43.056270 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:43.056270 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:43.056270 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 02:15:43.056270 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 02:15:43.056270 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 02:15:43.057381 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 02:15:43.057381 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:43.057381 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:43.057381 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:43.057381 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:43.057381 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:43.057381 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:43.057381 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:43.057381 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:43.057381 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:43.057381 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:43.057381 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:43.057381 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:43.057381 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:43.057381 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:43.057381 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:43.057381 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:43.058347 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 02:15:43.058347 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 02:15:43.058347 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 02:15:43.058347 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 02:15:43.058347 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 02:15:43.058347 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 02:15:43.058347 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 02:15:43.058347 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:15:43.058347 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 354/714] 10.222.0.66 () {56 vars in 1111 bytes} [Mon Jul 18 02:15:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:43.068669 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23b7579-162e-4c55-b251-883312fdcd84 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:43.134806 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:43.147852 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:43.160298 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744567af-3953-4edd-97fc-773593091886 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:43.207870 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68eb69d0-cd06-4576-822d-d62abd66cd14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:43.210174 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eb69d0-cd06-4576-822d-d62abd66cd14 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:43.210174 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eb69d0-cd06-4576-822d-d62abd66cd14 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:43.210174 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eb69d0-cd06-4576-822d-d62abd66cd14 None admin] PATH_INFO: `/v3/services/network` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:43.210578 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68eb69d0-cd06-4576-822d-d62abd66cd14 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:43.212501 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68eb69d0-cd06-4576-822d-d62abd66cd14 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:43.216085 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23b7579-162e-4c55-b251-883312fdcd84 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:43.231063 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-68eb69d0-cd06-4576-822d-d62abd66cd14 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 18 02:15:43.231063 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:43.231063 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 02:15:43.231063 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 02:15:43.231063 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 02:15:43.231063 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 02:15:43.231063 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 02:15:43.231063 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 02:15:43.231063 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 18 02:15:43.231063 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:15:43.231063 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 02:15:43.231063 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:15:43.231063 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:43.231063 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:43.231063 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:43.231063 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:43.231063 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:43.232134 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:43.232134 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:43.232134 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:43.232134 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:43.232134 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:43.232134 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:43.232134 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 02:15:43.232134 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 02:15:43.232134 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 02:15:43.232134 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 02:15:43.232134 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:43.232134 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:43.232134 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:43.232134 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:43.232134 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:43.232134 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:43.233022 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:43.233022 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:43.233022 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:43.233022 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:43.233022 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:43.233022 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:43.233022 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:43.233022 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:43.233022 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:43.233022 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:43.233022 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 02:15:43.233022 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 02:15:43.233022 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 18 02:15:43.233022 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:15:43.233791 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 361/715] 10.222.0.66 () {56 vars in 1111 bytes} [Mon Jul 18 02:15:42 2022] GET /identity/v3/services/network => generated 88 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:43.234736 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3949e178-1289-4e97-984d-20e704df7ae7 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:43.235022 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3949e178-1289-4e97-984d-20e704df7ae7 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:43.235299 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3949e178-1289-4e97-984d-20e704df7ae7 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:43.237031 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 362/716] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:15:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:15:43.241339 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3949e178-1289-4e97-984d-20e704df7ae7 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:43.259997 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824c5ed6-2da7-442f-9948-a5cd17bb4116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:43.260670 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824c5ed6-2da7-442f-9948-a5cd17bb4116 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:43.260955 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824c5ed6-2da7-442f-9948-a5cd17bb4116 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:43.261231 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824c5ed6-2da7-442f-9948-a5cd17bb4116 None admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:43.262722 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824c5ed6-2da7-442f-9948-a5cd17bb4116 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:43.263038 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824c5ed6-2da7-442f-9948-a5cd17bb4116 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:43.264504 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824c5ed6-2da7-442f-9948-a5cd17bb4116 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:43.277349 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 355/717] 10.222.0.66 () {56 vars in 1148 bytes} [Mon Jul 18 02:15:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:43.278881 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db42d7b-dfbf-47b2-9684-018caf4324d5 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:43.279237 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db42d7b-dfbf-47b2-9684-018caf4324d5 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:43.279448 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db42d7b-dfbf-47b2-9684-018caf4324d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:43.297318 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-3db42d7b-dfbf-47b2-9684-018caf4324d5 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:15:43.298523 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-3db42d7b-dfbf-47b2-9684-018caf4324d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:15:43.362243 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3949e178-1289-4e97-984d-20e704df7ae7 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:43.389722 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 356/718] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:15:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 02:15:43.390976 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db42d7b-dfbf-47b2-9684-018caf4324d5 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:43.412373 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd089bfd-2cee-42b5-bbab-71ef96f4e0bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:43.413051 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd089bfd-2cee-42b5-bbab-71ef96f4e0bb None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:43.413335 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd089bfd-2cee-42b5-bbab-71ef96f4e0bb None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:43.413614 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd089bfd-2cee-42b5-bbab-71ef96f4e0bb None admin] PATH_INFO: `/v3/services` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:43.414401 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd089bfd-2cee-42b5-bbab-71ef96f4e0bb None admin] RBAC: Adding query filter params (name=network) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:43.414712 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd089bfd-2cee-42b5-bbab-71ef96f4e0bb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:43.416099 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd089bfd-2cee-42b5-bbab-71ef96f4e0bb None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:43.426640 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 363/719] 10.222.0.66 () {56 vars in 1112 bytes} [Mon Jul 18 02:15:43 2022] GET /identity/v3/services?name=network => generated 124 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:43.427264 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:43.427542 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd089bfd-2cee-42b5-bbab-71ef96f4e0bb None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:43.447986 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:43.538765 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db42d7b-dfbf-47b2-9684-018caf4324d5 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:43.539114 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd089bfd-2cee-42b5-bbab-71ef96f4e0bb None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:43.574752 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f45a5c-0fb7-4da4-8a7c-e47c2077f926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:43.575797 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f45a5c-0fb7-4da4-8a7c-e47c2077f926 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:43.576081 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f45a5c-0fb7-4da4-8a7c-e47c2077f926 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:43.576355 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f45a5c-0fb7-4da4-8a7c-e47c2077f926 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:43.577682 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3525abef-45e0-49ac-9068-2cb0a2024ee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:43.578468 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3525abef-45e0-49ac-9068-2cb0a2024ee6 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:43.578753 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3525abef-45e0-49ac-9068-2cb0a2024ee6 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:43.579126 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3525abef-45e0-49ac-9068-2cb0a2024ee6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:43.579964 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3525abef-45e0-49ac-9068-2cb0a2024ee6 None admin] RBAC: Adding query filter params (role.id=d4aa3324f0c84887b41d0dc4acf7f30e, scope.project.id=76f7c0ae36ae49b09dcae4c3e9a875f4, user.id=80b138831bc54c7b986d50092d62d4af) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:43.580275 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3525abef-45e0-49ac-9068-2cb0a2024ee6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:43.581726 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3525abef-45e0-49ac-9068-2cb0a2024ee6 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:43.586909 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f45a5c-0fb7-4da4-8a7c-e47c2077f926 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 02:15:43.588361 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 357/720] 10.222.0.66 () {56 vars in 1374 bytes} [Mon Jul 18 02:15:43 2022] GET /identity/v3/role_assignments?user.id=80b138831bc54c7b986d50092d62d4af&scope.project.id=76f7c0ae36ae49b09dcae4c3e9a875f4&role.id=d4aa3324f0c84887b41d0dc4acf7f30e => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:43.588959 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3525abef-45e0-49ac-9068-2cb0a2024ee6 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:43.596069 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-a2f45a5c-0fb7-4da4-8a7c-e47c2077f926 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 02:15:43.596069 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:43.596069 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:43.596069 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:43.596069 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:43.596069 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:43.596069 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:43.596069 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:43.596069 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:43.596069 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:43.596069 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:43.596069 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:43.596069 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 02:15:43.596069 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 02:15:43.596069 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 02:15:43.596069 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 02:15:43.597094 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 02:15:43.597094 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 02:15:43.597094 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 02:15:43.597094 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 02:15:43.597094 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:43.597094 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:43.597094 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:43.597094 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:43.597094 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:43.597094 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:43.597094 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:43.597094 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:43.597094 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:43.597094 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:43.597094 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:43.597094 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:43.597993 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:43.597993 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:43.597993 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:43.597993 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:43.597993 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 02:15:43.597993 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 02:15:43.597993 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 02:15:43.597993 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 02:15:43.597993 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 02:15:43.597993 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 02:15:43.597993 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 02:15:43.597993 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:15:43.597993 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 364/721] 10.222.0.66 () {56 vars in 1102 bytes} [Mon Jul 18 02:15:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:43.606425 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f45a5c-0fb7-4da4-8a7c-e47c2077f926 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:43.695524 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:43.698195 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:43.713230 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3525abef-45e0-49ac-9068-2cb0a2024ee6 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:43.722970 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f45a5c-0fb7-4da4-8a7c-e47c2077f926 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:43.757540 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f796ce2-6e79-476e-a975-627b303ee37a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:43.758639 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f796ce2-6e79-476e-a975-627b303ee37a None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:43.758938 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f796ce2-6e79-476e-a975-627b303ee37a None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:43.759218 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f796ce2-6e79-476e-a975-627b303ee37a None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:43.760201 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f796ce2-6e79-476e-a975-627b303ee37a None admin] RBAC: Adding query filter params (type=network) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:43.760578 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f796ce2-6e79-476e-a975-627b303ee37a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:43.763130 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f796ce2-6e79-476e-a975-627b303ee37a None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:43.774117 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 358/722] 10.222.0.66 () {56 vars in 1112 bytes} [Mon Jul 18 02:15:43 2022] GET /identity/v3/services?type=network => generated 124 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:43.779485 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086b9651-3166-4826-8b08-7c6663e97bf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:43.780540 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086b9651-3166-4826-8b08-7c6663e97bf2 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:43.781026 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086b9651-3166-4826-8b08-7c6663e97bf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:43.781528 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086b9651-3166-4826-8b08-7c6663e97bf2 None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:43.782603 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086b9651-3166-4826-8b08-7c6663e97bf2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:43.783165 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086b9651-3166-4826-8b08-7c6663e97bf2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:43.784799 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086b9651-3166-4826-8b08-7c6663e97bf2 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:43.805765 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 365/723] 10.222.0.66 () {56 vars in 1103 bytes} [Mon Jul 18 02:15:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 02:15:43.811941 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f796ce2-6e79-476e-a975-627b303ee37a None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:43.959626 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f796ce2-6e79-476e-a975-627b303ee37a None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:44.002867 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a3e7e8-f1c2-4cc1-9615-94053dce69da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:44.004030 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a3e7e8-f1c2-4cc1-9615-94053dce69da None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:44.004512 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a3e7e8-f1c2-4cc1-9615-94053dce69da None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:44.005016 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a3e7e8-f1c2-4cc1-9615-94053dce69da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:44.019020 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a3e7e8-f1c2-4cc1-9615-94053dce69da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:44.020935 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a3e7e8-f1c2-4cc1-9615-94053dce69da None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:44.032059 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:44.042474 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-01a3e7e8-f1c2-4cc1-9615-94053dce69da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:15:44.042474 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:44.042474 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:44.042474 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:44.042474 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:44.042474 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:44.042474 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:44.042474 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:44.042474 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:44.042474 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:44.042474 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:44.042474 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:44.042474 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 02:15:44.042474 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 02:15:44.042474 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 02:15:44.051920 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 02:15:44.051920 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:44.051920 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:44.051920 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:44.051920 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:44.051920 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:44.051920 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:44.051920 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:44.051920 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:44.051920 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:44.051920 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:44.051920 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:44.051920 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:44.051920 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:44.051920 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:44.051920 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:44.051920 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:44.052893 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 02:15:44.052893 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 02:15:44.052893 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:15:44.052893 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:15:44.052893 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 359/724] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 18 02:15:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:44.054887 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086b9651-3166-4826-8b08-7c6663e97bf2 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:44.063200 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af90d17c-b505-46fa-bd11-2920f6b93043 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:44.063772 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af90d17c-b505-46fa-bd11-2920f6b93043 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:44.064267 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af90d17c-b505-46fa-bd11-2920f6b93043 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:44.065837 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 360/725] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:15:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:15:44.072610 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8911309c-5774-4b0e-9ca5-4de0c2d60d95 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:44.073432 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8911309c-5774-4b0e-9ca5-4de0c2d60d95 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:44.073432 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8911309c-5774-4b0e-9ca5-4de0c2d60d95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:44.096013 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-8911309c-5774-4b0e-9ca5-4de0c2d60d95 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:15:44.097174 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-8911309c-5774-4b0e-9ca5-4de0c2d60d95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:15:44.173219 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086b9651-3166-4826-8b08-7c6663e97bf2 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:44.207767 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a2331a-2606-4639-8fd0-1d294da9884f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:44.208798 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a2331a-2606-4639-8fd0-1d294da9884f None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:44.209289 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a2331a-2606-4639-8fd0-1d294da9884f None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:44.209757 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a2331a-2606-4639-8fd0-1d294da9884f None admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:44.210704 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a2331a-2606-4639-8fd0-1d294da9884f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:44.211207 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a2331a-2606-4639-8fd0-1d294da9884f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:44.212767 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a2331a-2606-4639-8fd0-1d294da9884f None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:44.223750 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 361/726] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:15:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 02:15:44.226877 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 366/727] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 18 02:15:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:44.227520 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:44.234230 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8911309c-5774-4b0e-9ca5-4de0c2d60d95 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:44.275143 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a2331a-2606-4639-8fd0-1d294da9884f None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:44.393809 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a2331a-2606-4639-8fd0-1d294da9884f None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:44.505889 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8911309c-5774-4b0e-9ca5-4de0c2d60d95 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:44.506930 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6879b3d2-3308-41f8-9b69-b318da527897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:44.507958 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6879b3d2-3308-41f8-9b69-b318da527897 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:44.509856 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6879b3d2-3308-41f8-9b69-b318da527897 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:44.510374 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6879b3d2-3308-41f8-9b69-b318da527897 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:44.537337 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6879b3d2-3308-41f8-9b69-b318da527897 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 02:15:44.545805 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-6879b3d2-3308-41f8-9b69-b318da527897 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 02:15:44.545805 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:44.545805 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:44.545805 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:44.545805 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:44.545805 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:44.545805 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:44.545805 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:44.545805 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:44.545805 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:44.545805 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:44.545805 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:44.545805 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 02:15:44.545805 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 02:15:44.545805 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 02:15:44.545805 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 02:15:44.547496 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 02:15:44.547496 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 02:15:44.547496 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 02:15:44.547496 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 02:15:44.547496 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:44.547496 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:44.547496 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:44.547496 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:44.547496 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:44.547496 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:44.547496 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:44.547496 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:44.547496 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:44.547496 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:44.547496 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:44.547496 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:44.548343 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:44.548343 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:44.548343 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:44.548343 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:44.548343 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 02:15:44.548343 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 02:15:44.548343 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 02:15:44.548343 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 02:15:44.548343 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 02:15:44.548343 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 02:15:44.548343 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 02:15:44.548343 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:15:44.555525 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 367/728] 10.222.0.66 () {56 vars in 1096 bytes} [Mon Jul 18 02:15:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:44.555858 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:44.563670 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6879b3d2-3308-41f8-9b69-b318da527897 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:44.568389 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05981be7-fef3-45b8-920c-64db134a117a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:44.569379 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05981be7-fef3-45b8-920c-64db134a117a None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:44.569813 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05981be7-fef3-45b8-920c-64db134a117a None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:44.570269 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05981be7-fef3-45b8-920c-64db134a117a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:44.586746 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05981be7-fef3-45b8-920c-64db134a117a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:44.588223 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05981be7-fef3-45b8-920c-64db134a117a None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:44.616548 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-05981be7-fef3-45b8-920c-64db134a117a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 02:15:44.616548 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:44.616548 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:44.616548 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:44.616548 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:44.616548 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:44.616548 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:44.616548 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:44.616548 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:44.616548 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:44.616548 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:44.616548 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:44.616548 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 02:15:44.616548 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 02:15:44.616548 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 02:15:44.616548 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 02:15:44.617737 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:44.617737 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:44.617737 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 02:15:44.617737 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 02:15:44.617737 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 02:15:44.617737 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 02:15:44.617737 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:44.617737 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:44.617737 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:44.617737 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:44.617737 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:44.617737 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:44.617737 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:44.617737 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:44.617737 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:44.617737 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:44.617737 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:44.617737 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:44.619125 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:44.619125 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:44.619125 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 02:15:44.619125 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 02:15:44.619125 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 02:15:44.619125 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 02:15:44.619125 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 02:15:44.619125 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 02:15:44.619125 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 02:15:44.619125 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:15:44.619125 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 362/729] 10.222.0.66 () {56 vars in 1117 bytes} [Mon Jul 18 02:15:44 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 386 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:44.623911 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05981be7-fef3-45b8-920c-64db134a117a None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:44.719868 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:44.737607 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05981be7-fef3-45b8-920c-64db134a117a None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:44.800102 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6879b3d2-3308-41f8-9b69-b318da527897 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:44.812981 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f75e0a7-358a-481e-90ae-68f9976dfe0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:44.813669 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f75e0a7-358a-481e-90ae-68f9976dfe0d None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:44.813954 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f75e0a7-358a-481e-90ae-68f9976dfe0d None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:44.814273 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f75e0a7-358a-481e-90ae-68f9976dfe0d None admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:44.815117 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f75e0a7-358a-481e-90ae-68f9976dfe0d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:44.815430 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f75e0a7-358a-481e-90ae-68f9976dfe0d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:44.816871 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f75e0a7-358a-481e-90ae-68f9976dfe0d None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:44.863098 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 363/730] 10.222.0.66 () {56 vars in 1149 bytes} [Mon Jul 18 02:15:44 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:44.868899 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f75e0a7-358a-481e-90ae-68f9976dfe0d None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:44.877337 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c0af45-9bed-48c5-85a0-03ed8e8d24d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:44.878021 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c0af45-9bed-48c5-85a0-03ed8e8d24d2 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:44.878329 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c0af45-9bed-48c5-85a0-03ed8e8d24d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:44.878607 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c0af45-9bed-48c5-85a0-03ed8e8d24d2 None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:44.879362 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c0af45-9bed-48c5-85a0-03ed8e8d24d2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:44.879670 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c0af45-9bed-48c5-85a0-03ed8e8d24d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:44.883522 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c0af45-9bed-48c5-85a0-03ed8e8d24d2 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:44.900205 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 368/731] 10.222.0.66 () {56 vars in 1099 bytes} [Mon Jul 18 02:15:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:44.905111 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c0af45-9bed-48c5-85a0-03ed8e8d24d2 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:44.968667 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:45.012429 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c0af45-9bed-48c5-85a0-03ed8e8d24d2 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:45.046461 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:45.050726 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574ee015-7387-4c43-ba3f-6a1a6519606f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:45.051589 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574ee015-7387-4c43-ba3f-6a1a6519606f None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:45.051865 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574ee015-7387-4c43-ba3f-6a1a6519606f None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:45.052147 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574ee015-7387-4c43-ba3f-6a1a6519606f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:45.064904 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574ee015-7387-4c43-ba3f-6a1a6519606f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:45.066536 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574ee015-7387-4c43-ba3f-6a1a6519606f None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:45.088022 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-574ee015-7387-4c43-ba3f-6a1a6519606f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:15:45.088022 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:45.088022 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:45.088022 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:45.088022 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:45.088022 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:45.088022 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:45.088022 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:45.088022 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:45.088022 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:45.088022 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:45.088022 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:45.088022 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 02:15:45.088022 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 02:15:45.088022 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 02:15:45.088981 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 02:15:45.088981 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:45.088981 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:45.088981 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:45.088981 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:45.088981 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:45.088981 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:45.088981 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:45.088981 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:45.088981 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:45.088981 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:45.088981 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:45.088981 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:45.088981 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:45.088981 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:45.088981 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:45.088981 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:45.090555 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 02:15:45.090555 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 02:15:45.090555 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:15:45.090555 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:15:45.090555 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 369/732] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 18 02:15:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:45.097698 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574ee015-7387-4c43-ba3f-6a1a6519606f None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:45.113881 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f75e0a7-358a-481e-90ae-68f9976dfe0d None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:45.178953 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b9efb3-8e2b-4f71-a4fa-9475fcb08f91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:45.179615 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b9efb3-8e2b-4f71-a4fa-9475fcb08f91 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:45.179900 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b9efb3-8e2b-4f71-a4fa-9475fcb08f91 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:45.180178 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b9efb3-8e2b-4f71-a4fa-9475fcb08f91 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:45.298357 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b9efb3-8e2b-4f71-a4fa-9475fcb08f91 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:45.299768 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b9efb3-8e2b-4f71-a4fa-9475fcb08f91 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:45.306787 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574ee015-7387-4c43-ba3f-6a1a6519606f None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:45.368714 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81523195-7547-4ec1-b7f5-ceab0232ea14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:45.370026 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81523195-7547-4ec1-b7f5-ceab0232ea14 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:45.372706 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81523195-7547-4ec1-b7f5-ceab0232ea14 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:45.373194 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81523195-7547-4ec1-b7f5-ceab0232ea14 None admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:45.374154 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81523195-7547-4ec1-b7f5-ceab0232ea14 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:45.374666 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81523195-7547-4ec1-b7f5-ceab0232ea14 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:45.376885 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81523195-7547-4ec1-b7f5-ceab0232ea14 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:45.403994 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:45.404315 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 370/733] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 18 02:15:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:45.405953 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aa4a88-b09f-4902-8b55-99b9ccbea4df None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:45.412639 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aa4a88-b09f-4902-8b55-99b9ccbea4df None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:45.412639 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aa4a88-b09f-4902-8b55-99b9ccbea4df None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:45.412639 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 371/734] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:15:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:15:45.414503 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f2c9a1-f748-43b7-99f8-b6014691d2f7 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:45.414997 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f2c9a1-f748-43b7-99f8-b6014691d2f7 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:45.415465 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f2c9a1-f748-43b7-99f8-b6014691d2f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:45.465512 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-f9f2c9a1-f748-43b7-99f8-b6014691d2f7 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:15:45.467312 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-f9f2c9a1-f748-43b7-99f8-b6014691d2f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:15:45.489963 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 364/735] 10.222.0.66 () {58 vars in 1282 bytes} [Mon Jul 18 02:15:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 621 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 18 02:15:45.491099 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b9efb3-8e2b-4f71-a4fa-9475fcb08f91 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:45.496951 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:45.550249 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 372/736] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:15:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 02:15:45.550249 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f2c9a1-f748-43b7-99f8-b6014691d2f7 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:45.630295 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b9efb3-8e2b-4f71-a4fa-9475fcb08f91 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:45.673395 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacc90e0-f503-4993-9023-f083da9d2f74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:45.674373 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacc90e0-f503-4993-9023-f083da9d2f74 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:45.674652 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacc90e0-f503-4993-9023-f083da9d2f74 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:45.674923 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacc90e0-f503-4993-9023-f083da9d2f74 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:45.684507 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacc90e0-f503-4993-9023-f083da9d2f74 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:45.686134 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacc90e0-f503-4993-9023-f083da9d2f74 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:45.696996 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f2c9a1-f748-43b7-99f8-b6014691d2f7 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:45.705103 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-eacc90e0-f503-4993-9023-f083da9d2f74 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 18 02:15:45.705103 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:45.705103 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:45.705103 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:45.705103 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:45.705103 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:45.705103 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:45.705103 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:45.705103 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:45.705103 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:45.705103 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:45.705103 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:45.705103 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 02:15:45.705103 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 02:15:45.705103 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 02:15:45.705103 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 02:15:45.706107 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:45.706107 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:45.706107 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 02:15:45.706107 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 02:15:45.706107 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 02:15:45.706107 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 02:15:45.706107 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:45.706107 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:45.706107 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:45.706107 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:45.706107 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:45.706107 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:45.706107 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:45.706107 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:45.706107 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:45.706107 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:45.706107 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:45.706107 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:45.707133 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:45.707133 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:45.707133 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 02:15:45.707133 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 02:15:45.707133 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 02:15:45.707133 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 02:15:45.707133 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 02:15:45.707133 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 02:15:45.707133 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 18 02:15:45.707133 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:15:45.710163 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 365/737] 10.222.0.66 () {56 vars in 1096 bytes} [Mon Jul 18 02:15:45 2022] GET /identity/v3/users/swift => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:45.711189 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacc90e0-f503-4993-9023-f083da9d2f74 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:45.727745 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:45.751857 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89be4f18-083e-4473-af43-361a3df7a9da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:45.752648 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89be4f18-083e-4473-af43-361a3df7a9da None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:45.752928 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89be4f18-083e-4473-af43-361a3df7a9da None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:45.753199 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89be4f18-083e-4473-af43-361a3df7a9da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:45.769930 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89be4f18-083e-4473-af43-361a3df7a9da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:45.772775 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89be4f18-083e-4473-af43-361a3df7a9da None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:45.798898 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-89be4f18-083e-4473-af43-361a3df7a9da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:15:45.798898 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:45.798898 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:45.798898 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:45.798898 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:45.798898 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:45.798898 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:45.798898 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:45.798898 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:45.798898 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:45.798898 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:45.798898 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:45.798898 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 02:15:45.798898 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 02:15:45.798898 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 02:15:45.799891 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 02:15:45.799891 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:45.799891 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:45.799891 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:45.799891 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:45.799891 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:45.799891 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:45.799891 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:45.799891 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:45.799891 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:45.799891 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:45.799891 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:45.799891 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:45.799891 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:45.799891 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:45.799891 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:45.799891 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:45.800872 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 02:15:45.800872 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 02:15:45.800872 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:15:45.800872 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:15:45.804292 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 373/738] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 18 02:15:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:45.805494 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89be4f18-083e-4473-af43-361a3df7a9da None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:45.829443 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:45.844697 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacc90e0-f503-4993-9023-f083da9d2f74 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:45.888725 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41af5ee-1190-496a-81de-01928cd7cd83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:45.889532 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41af5ee-1190-496a-81de-01928cd7cd83 None admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:45.889817 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41af5ee-1190-496a-81de-01928cd7cd83 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:45.890115 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41af5ee-1190-496a-81de-01928cd7cd83 None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:45.890843 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41af5ee-1190-496a-81de-01928cd7cd83 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:45.892730 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41af5ee-1190-496a-81de-01928cd7cd83 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:45.934108 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89be4f18-083e-4473-af43-361a3df7a9da None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:45.971588 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0926916c-5d70-4c03-b80b-d47be8581cb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:45.972344 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0926916c-5d70-4c03-b80b-d47be8581cb2 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:45.972637 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0926916c-5d70-4c03-b80b-d47be8581cb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:45.972916 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0926916c-5d70-4c03-b80b-d47be8581cb2 None admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:45.973758 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0926916c-5d70-4c03-b80b-d47be8581cb2 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:45.974073 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0926916c-5d70-4c03-b80b-d47be8581cb2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:45.975547 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0926916c-5d70-4c03-b80b-d47be8581cb2 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:45.979954 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 366/739] 10.222.0.66 () {60 vars in 1141 bytes} [Mon Jul 18 02:15:45 2022] POST /identity/v3/services => generated 251 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:15:45.980855 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41af5ee-1190-496a-81de-01928cd7cd83 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:46.009876 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 374/740] 10.222.0.66 () {56 vars in 1135 bytes} [Mon Jul 18 02:15:45 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:46.012774 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3b6bbc-4a13-40fc-8098-37dc44e65e89 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:46.013331 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3b6bbc-4a13-40fc-8098-37dc44e65e89 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:46.013976 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3b6bbc-4a13-40fc-8098-37dc44e65e89 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:46.016495 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 375/741] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:15:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:15:46.021150 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3b6bbc-4a13-40fc-8098-37dc44e65e89 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:46.075518 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:46.128570 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:46.138190 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41af5ee-1190-496a-81de-01928cd7cd83 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:46.164619 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3b6bbc-4a13-40fc-8098-37dc44e65e89 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:46.189534 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3af029-da6f-42ed-b8ff-03a1b4ddfdd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:46.190242 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3af029-da6f-42ed-b8ff-03a1b4ddfdd1 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:46.190526 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3af029-da6f-42ed-b8ff-03a1b4ddfdd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:46.190799 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3af029-da6f-42ed-b8ff-03a1b4ddfdd1 None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:46.191521 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3af029-da6f-42ed-b8ff-03a1b4ddfdd1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:46.191824 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3af029-da6f-42ed-b8ff-03a1b4ddfdd1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:46.193316 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3af029-da6f-42ed-b8ff-03a1b4ddfdd1 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:46.209213 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 367/742] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 18 02:15:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:46.209800 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672cfc79-c281-4e9a-abc6-acacb47c43e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:46.210528 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ab7900-1365-48c7-95b8-31fd15f6755b None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:46.210643 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672cfc79-c281-4e9a-abc6-acacb47c43e0 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:46.210839 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ab7900-1365-48c7-95b8-31fd15f6755b None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:46.210943 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672cfc79-c281-4e9a-abc6-acacb47c43e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:46.211111 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ab7900-1365-48c7-95b8-31fd15f6755b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:46.211205 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672cfc79-c281-4e9a-abc6-acacb47c43e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:46.235123 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-02ab7900-1365-48c7-95b8-31fd15f6755b None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:15:46.236529 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-02ab7900-1365-48c7-95b8-31fd15f6755b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:15:46.293768 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672cfc79-c281-4e9a-abc6-acacb47c43e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:46.295440 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672cfc79-c281-4e9a-abc6-acacb47c43e0 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:46.312878 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 368/743] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:15:46 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 02:15:46.314671 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ab7900-1365-48c7-95b8-31fd15f6755b None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:46.378375 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:46.380708 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:46.409460 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 376/744] 10.222.0.66 () {58 vars in 1282 bytes} [Mon Jul 18 02:15:46 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 18 02:15:46.410924 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970db47f-4f17-49d8-a98c-64d3727a6d87 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:46.411284 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970db47f-4f17-49d8-a98c-64d3727a6d87 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:46.411570 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970db47f-4f17-49d8-a98c-64d3727a6d87 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:46.412888 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 377/745] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:15:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:15:46.417771 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970db47f-4f17-49d8-a98c-64d3727a6d87 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:46.431883 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ab7900-1365-48c7-95b8-31fd15f6755b None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:46.469885 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f3b4e7-ea5f-4828-9b99-90bad3b73988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:46.470713 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f3b4e7-ea5f-4828-9b99-90bad3b73988 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:46.470991 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f3b4e7-ea5f-4828-9b99-90bad3b73988 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:46.471260 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f3b4e7-ea5f-4828-9b99-90bad3b73988 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:46.478433 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f3b4e7-ea5f-4828-9b99-90bad3b73988 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:46.479913 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f3b4e7-ea5f-4828-9b99-90bad3b73988 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:46.529936 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970db47f-4f17-49d8-a98c-64d3727a6d87 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:46.570658 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4100e878-8179-45ce-a0af-e8601b86de2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:46.571495 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4100e878-8179-45ce-a0af-e8601b86de2f None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:46.571744 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4100e878-8179-45ce-a0af-e8601b86de2f None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:46.572017 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4100e878-8179-45ce-a0af-e8601b86de2f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:46.585334 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4100e878-8179-45ce-a0af-e8601b86de2f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 02:15:46.593787 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-4100e878-8179-45ce-a0af-e8601b86de2f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 02:15:46.593787 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:46.593787 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:46.593787 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:46.593787 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:46.593787 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:46.593787 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:46.593787 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:46.593787 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:46.593787 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:46.593787 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:46.593787 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:46.593787 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 02:15:46.593787 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 02:15:46.593787 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 02:15:46.593787 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 02:15:46.594846 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 02:15:46.594846 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 02:15:46.594846 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 02:15:46.594846 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 02:15:46.594846 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:46.594846 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:46.594846 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:46.594846 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:46.594846 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:46.594846 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:46.594846 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:46.594846 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:46.594846 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:46.594846 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:46.594846 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:46.594846 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:46.595722 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:46.595722 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:46.595722 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:46.595722 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:46.595722 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 02:15:46.595722 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 02:15:46.595722 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 02:15:46.595722 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 02:15:46.595722 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 02:15:46.595722 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 02:15:46.595722 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 02:15:46.595722 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:15:46.600953 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 378/746] 10.222.0.66 () {56 vars in 1096 bytes} [Mon Jul 18 02:15:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:46.601181 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4100e878-8179-45ce-a0af-e8601b86de2f None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:46.611671 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-05f3b4e7-ea5f-4828-9b99-90bad3b73988 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 02:15:46.611671 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:46.611671 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:46.611671 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:46.611671 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:46.611671 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:46.611671 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:46.611671 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:46.611671 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:46.611671 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:46.611671 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:46.611671 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:46.611671 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 02:15:46.611671 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 02:15:46.611671 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 02:15:46.612548 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 02:15:46.612548 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:46.612548 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:46.612548 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:46.612548 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:46.612548 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:46.612548 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:46.612548 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:46.612548 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:46.612548 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:46.612548 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:46.612548 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:46.612548 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:46.612548 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:46.612548 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:46.612548 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:46.612548 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:46.613439 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 02:15:46.613439 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 02:15:46.613439 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 02:15:46.613439 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 02:15:46.613439 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 02:15:46.613439 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 02:15:46.613439 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 02:15:46.613439 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:15:46.617611 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 369/747] 10.222.0.66 () {56 vars in 1111 bytes} [Mon Jul 18 02:15:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:46.618701 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990c5796-f2e1-4685-bd6e-152904cd4fbd None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:46.618988 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990c5796-f2e1-4685-bd6e-152904cd4fbd None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:46.619263 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990c5796-f2e1-4685-bd6e-152904cd4fbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:46.635985 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-990c5796-f2e1-4685-bd6e-152904cd4fbd None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:15:46.637645 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-990c5796-f2e1-4685-bd6e-152904cd4fbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:15:46.640697 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:46.709109 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4100e878-8179-45ce-a0af-e8601b86de2f None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:46.715859 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 370/748] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:15:46 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 02:15:46.717356 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990c5796-f2e1-4685-bd6e-152904cd4fbd None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:46.744850 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e825ba-1fdb-4c45-b7b4-0a276e57a9b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:46.745650 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e825ba-1fdb-4c45-b7b4-0a276e57a9b3 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:46.746486 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e825ba-1fdb-4c45-b7b4-0a276e57a9b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:46.746764 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e825ba-1fdb-4c45-b7b4-0a276e57a9b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:46.757959 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e825ba-1fdb-4c45-b7b4-0a276e57a9b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:46.759541 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e825ba-1fdb-4c45-b7b4-0a276e57a9b3 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:46.777582 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:46.782070 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-74e825ba-1fdb-4c45-b7b4-0a276e57a9b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:15:46.782070 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:46.782070 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:46.782070 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:46.782070 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:46.782070 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:46.782070 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:46.782070 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:46.782070 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:46.782070 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:46.782070 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:46.782070 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:46.782070 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 02:15:46.782070 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 02:15:46.782070 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 02:15:46.783084 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 02:15:46.783084 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:46.783084 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:46.783084 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:46.783084 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:46.783084 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:46.783084 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:46.783084 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:46.783084 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:46.783084 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:46.783084 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:46.783084 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:46.783084 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:46.783084 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:46.783084 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:46.783084 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:46.783084 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:46.788949 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 02:15:46.788949 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 02:15:46.788949 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:15:46.788949 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:15:46.788949 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 379/749] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 18 02:15:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:46.788949 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e825ba-1fdb-4c45-b7b4-0a276e57a9b3 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:46.833366 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990c5796-f2e1-4685-bd6e-152904cd4fbd None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:46.867348 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4a4ff2-0df3-4e79-b4cb-d4c577615f18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:46.868019 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4a4ff2-0df3-4e79-b4cb-d4c577615f18 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:46.868303 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4a4ff2-0df3-4e79-b4cb-d4c577615f18 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:46.868584 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4a4ff2-0df3-4e79-b4cb-d4c577615f18 None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:46.869327 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4a4ff2-0df3-4e79-b4cb-d4c577615f18 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:46.869636 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4a4ff2-0df3-4e79-b4cb-d4c577615f18 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:46.871645 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4a4ff2-0df3-4e79-b4cb-d4c577615f18 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:46.880498 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 371/750] 10.222.0.66 () {56 vars in 1099 bytes} [Mon Jul 18 02:15:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:46.881184 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4a4ff2-0df3-4e79-b4cb-d4c577615f18 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:46.891128 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:46.906923 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e825ba-1fdb-4c45-b7b4-0a276e57a9b3 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:46.943594 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930b43c8-9bec-49da-881d-82c85627d04e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:46.944255 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930b43c8-9bec-49da-881d-82c85627d04e None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:46.944540 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930b43c8-9bec-49da-881d-82c85627d04e None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:46.944951 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930b43c8-9bec-49da-881d-82c85627d04e None admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:46.945594 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930b43c8-9bec-49da-881d-82c85627d04e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:46.945907 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930b43c8-9bec-49da-881d-82c85627d04e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:46.947447 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930b43c8-9bec-49da-881d-82c85627d04e None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:46.966169 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 380/751] 10.222.0.66 () {56 vars in 1148 bytes} [Mon Jul 18 02:15:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:46.966965 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930b43c8-9bec-49da-881d-82c85627d04e None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:46.991891 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4a4ff2-0df3-4e79-b4cb-d4c577615f18 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:46.999857 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:47.027972 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4076cef9-cc7b-4ecc-8e72-492caae908a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:47.028639 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4076cef9-cc7b-4ecc-8e72-492caae908a1 None admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:47.028922 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4076cef9-cc7b-4ecc-8e72-492caae908a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:47.029195 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4076cef9-cc7b-4ecc-8e72-492caae908a1 None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:47.029920 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4076cef9-cc7b-4ecc-8e72-492caae908a1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:47.031330 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4076cef9-cc7b-4ecc-8e72-492caae908a1 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:47.070426 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930b43c8-9bec-49da-881d-82c85627d04e None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:47.104021 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7fad1c-bc47-4689-8e4d-9ed0e7fb46de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:47.104678 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7fad1c-bc47-4689-8e4d-9ed0e7fb46de None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:47.104958 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7fad1c-bc47-4689-8e4d-9ed0e7fb46de None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:47.105934 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7fad1c-bc47-4689-8e4d-9ed0e7fb46de None admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:47.106803 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7fad1c-bc47-4689-8e4d-9ed0e7fb46de None admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:47.107113 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7fad1c-bc47-4689-8e4d-9ed0e7fb46de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:47.108532 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7fad1c-bc47-4689-8e4d-9ed0e7fb46de None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:47.119812 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 381/752] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 18 02:15:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:47.120634 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7fad1c-bc47-4689-8e4d-9ed0e7fb46de None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:47.127794 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 372/753] 10.222.0.66 () {60 vars in 1141 bytes} [Mon Jul 18 02:15:46 2022] POST /identity/v3/services => generated 240 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:15:47.128870 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4076cef9-cc7b-4ecc-8e72-492caae908a1 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:47.133166 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:47.263473 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7fad1c-bc47-4689-8e4d-9ed0e7fb46de None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:47.284780 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4076cef9-cc7b-4ecc-8e72-492caae908a1 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:47.311132 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c27a6d5-e08f-4581-9619-de99c441000c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:47.312269 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c27a6d5-e08f-4581-9619-de99c441000c None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:47.312788 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c27a6d5-e08f-4581-9619-de99c441000c None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:47.313137 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c27a6d5-e08f-4581-9619-de99c441000c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:47.326846 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c27a6d5-e08f-4581-9619-de99c441000c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:47.328571 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c27a6d5-e08f-4581-9619-de99c441000c None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:47.332684 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e02df71-081a-46a0-990d-02e272a3462c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:47.333354 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e02df71-081a-46a0-990d-02e272a3462c None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:47.333638 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e02df71-081a-46a0-990d-02e272a3462c None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:47.333914 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e02df71-081a-46a0-990d-02e272a3462c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:47.334852 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e02df71-081a-46a0-990d-02e272a3462c None admin] RBAC: Adding query filter params (role.id=eea94daf3312421cb12aa8abfa73160e, scope.project.id=76f7c0ae36ae49b09dcae4c3e9a875f4, user.id=768f872096584d4093fc616d547aabb6) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:47.335120 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e02df71-081a-46a0-990d-02e272a3462c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:47.336547 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e02df71-081a-46a0-990d-02e272a3462c None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:47.345122 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 373/754] 10.222.0.66 () {56 vars in 1374 bytes} [Mon Jul 18 02:15:47 2022] GET /identity/v3/role_assignments?user.id=768f872096584d4093fc616d547aabb6&scope.project.id=76f7c0ae36ae49b09dcae4c3e9a875f4&role.id=eea94daf3312421cb12aa8abfa73160e => generated 259 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:47.345928 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e02df71-081a-46a0-990d-02e272a3462c None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:47.354550 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-5c27a6d5-e08f-4581-9619-de99c441000c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:15:47.354550 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:47.354550 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:47.354550 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:47.354550 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:47.354550 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:47.354550 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:47.354550 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:47.354550 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:47.354550 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:47.354550 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:47.354550 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:47.354550 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 02:15:47.354550 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 02:15:47.354550 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 02:15:47.356120 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 02:15:47.356120 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:47.356120 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:47.356120 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:47.356120 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:47.356120 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:47.356120 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:47.356120 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:47.356120 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:47.356120 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:47.356120 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:47.356120 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:47.356120 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:47.356120 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:47.356120 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:47.356120 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:47.356120 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:47.357031 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 02:15:47.357031 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 02:15:47.357031 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:15:47.357031 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:15:47.357031 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 382/755] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 18 02:15:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:47.363221 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c27a6d5-e08f-4581-9619-de99c441000c None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:47.388095 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:47.438406 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:47.497983 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c27a6d5-e08f-4581-9619-de99c441000c None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:47.534698 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e02df71-081a-46a0-990d-02e272a3462c None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:47.559387 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52470b9-efed-4cb9-b9d1-4df4d641eddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:47.560066 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52470b9-efed-4cb9-b9d1-4df4d641eddd None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:47.560351 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52470b9-efed-4cb9-b9d1-4df4d641eddd None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:47.560631 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52470b9-efed-4cb9-b9d1-4df4d641eddd None admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:47.561358 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52470b9-efed-4cb9-b9d1-4df4d641eddd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:47.561665 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52470b9-efed-4cb9-b9d1-4df4d641eddd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:47.563092 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52470b9-efed-4cb9-b9d1-4df4d641eddd None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:47.585456 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 383/756] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 18 02:15:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:47.593463 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52470b9-efed-4cb9-b9d1-4df4d641eddd None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:47.606887 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d895382-9f6e-40d2-806c-670dd659c5cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:47.607702 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d895382-9f6e-40d2-806c-670dd659c5cf None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:47.607985 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d895382-9f6e-40d2-806c-670dd659c5cf None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:47.608259 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d895382-9f6e-40d2-806c-670dd659c5cf None admin] PATH_INFO: `/v3/projects/service` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:47.621890 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d895382-9f6e-40d2-806c-670dd659c5cf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:47.623401 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d895382-9f6e-40d2-806c-670dd659c5cf None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:47.647062 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-1d895382-9f6e-40d2-806c-670dd659c5cf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 02:15:47.647062 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:47.647062 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:47.647062 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:47.647062 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:47.647062 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:47.647062 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:47.647062 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:47.647062 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:47.647062 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:47.647062 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:47.647062 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:47.647062 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 02:15:47.647062 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 02:15:47.647062 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 02:15:47.648080 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 02:15:47.648080 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:47.648080 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:47.648080 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:47.648080 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:47.648080 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:47.648080 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:47.648080 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:47.648080 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:47.648080 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:47.648080 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:47.648080 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:47.648080 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:47.648080 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:47.648080 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:47.648080 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:47.648080 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:47.648991 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 02:15:47.648991 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 02:15:47.648991 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 02:15:47.648991 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 02:15:47.648991 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 02:15:47.648991 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 02:15:47.648991 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 02:15:47.648991 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:15:47.648991 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 374/757] 10.222.0.66 () {56 vars in 1111 bytes} [Mon Jul 18 02:15:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:47.657645 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d895382-9f6e-40d2-806c-670dd659c5cf None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:47.763666 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52470b9-efed-4cb9-b9d1-4df4d641eddd None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:47.769478 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:47.792320 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d895382-9f6e-40d2-806c-670dd659c5cf None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:47.812512 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:47.829891 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe5190a-91d5-480c-8ca1-b99bec1f0a6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:47.830730 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe5190a-91d5-480c-8ca1-b99bec1f0a6c None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:47.831598 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe5190a-91d5-480c-8ca1-b99bec1f0a6c None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:47.831884 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe5190a-91d5-480c-8ca1-b99bec1f0a6c None admin] PATH_INFO: `/v3/users/nova` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:47.840886 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe5190a-91d5-480c-8ca1-b99bec1f0a6c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:47.843774 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe5190a-91d5-480c-8ca1-b99bec1f0a6c None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:47.862427 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-5fe5190a-91d5-480c-8ca1-b99bec1f0a6c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 18 02:15:47.862427 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:47.862427 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:47.862427 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:47.862427 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:47.862427 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:47.862427 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:47.862427 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:47.862427 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:47.862427 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:47.862427 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:47.862427 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:47.862427 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 02:15:47.862427 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 02:15:47.862427 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 02:15:47.862427 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 02:15:47.864974 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:47.864974 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:47.864974 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 02:15:47.864974 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 02:15:47.864974 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 02:15:47.864974 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 02:15:47.864974 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:47.864974 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:47.864974 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:47.864974 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:47.864974 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:47.864974 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:47.864974 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:47.864974 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:47.864974 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:47.864974 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:47.864974 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:47.864974 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:47.867462 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:47.867462 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:47.867462 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 02:15:47.867462 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 02:15:47.867462 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 02:15:47.867462 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 02:15:47.867462 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 02:15:47.867462 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 02:15:47.867462 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 18 02:15:47.867462 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:15:47.868741 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acf8e74-d713-4082-9288-720218e1b27f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:47.868741 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acf8e74-d713-4082-9288-720218e1b27f None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:47.868741 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acf8e74-d713-4082-9288-720218e1b27f None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:47.868741 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acf8e74-d713-4082-9288-720218e1b27f None admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:47.869313 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acf8e74-d713-4082-9288-720218e1b27f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:47.869313 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acf8e74-d713-4082-9288-720218e1b27f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:47.869584 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 384/758] 10.222.0.66 () {56 vars in 1093 bytes} [Mon Jul 18 02:15:47 2022] GET /identity/v3/users/nova => generated 82 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:47.869749 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acf8e74-d713-4082-9288-720218e1b27f None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:47.869986 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe5190a-91d5-480c-8ca1-b99bec1f0a6c None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:47.899371 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 375/759] 10.222.0.66 () {56 vars in 1148 bytes} [Mon Jul 18 02:15:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 02:15:47.906497 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acf8e74-d713-4082-9288-720218e1b27f None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:48.023605 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acf8e74-d713-4082-9288-720218e1b27f None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:48.062859 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe5190a-91d5-480c-8ca1-b99bec1f0a6c None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:48.094867 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9dc0c7-fe1e-4153-85b6-763838f6ba57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:48.095531 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9dc0c7-fe1e-4153-85b6-763838f6ba57 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:48.095814 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9dc0c7-fe1e-4153-85b6-763838f6ba57 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:48.096091 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9dc0c7-fe1e-4153-85b6-763838f6ba57 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:48.096925 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9dc0c7-fe1e-4153-85b6-763838f6ba57 None admin] RBAC: Adding query filter params (role.id=d4aa3324f0c84887b41d0dc4acf7f30e, scope.project.id=76f7c0ae36ae49b09dcae4c3e9a875f4, user.id=b4300622cbd347dd8f5227bb960e98d0) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:48.097236 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9dc0c7-fe1e-4153-85b6-763838f6ba57 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:48.098721 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9dc0c7-fe1e-4153-85b6-763838f6ba57 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:48.109122 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 376/760] 10.222.0.66 () {56 vars in 1374 bytes} [Mon Jul 18 02:15:47 2022] GET /identity/v3/role_assignments?user.id=b4300622cbd347dd8f5227bb960e98d0&scope.project.id=76f7c0ae36ae49b09dcae4c3e9a875f4&role.id=d4aa3324f0c84887b41d0dc4acf7f30e => generated 259 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:48.119148 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77dbdda-4bd9-415d-ab09-5d56111ebebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:48.119852 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77dbdda-4bd9-415d-ab09-5d56111ebebd None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:48.120139 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77dbdda-4bd9-415d-ab09-5d56111ebebd None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:48.120419 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77dbdda-4bd9-415d-ab09-5d56111ebebd None admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:48.121381 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77dbdda-4bd9-415d-ab09-5d56111ebebd None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:48.121692 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77dbdda-4bd9-415d-ab09-5d56111ebebd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:48.123220 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77dbdda-4bd9-415d-ab09-5d56111ebebd None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:48.155770 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 385/761] 10.222.0.66 () {56 vars in 1133 bytes} [Mon Jul 18 02:15:47 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:48.162458 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9dc0c7-fe1e-4153-85b6-763838f6ba57 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:48.182033 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:48.281203 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fe1563-a1d7-4c5f-bca1-f410b9341dc3 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:48.281564 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fe1563-a1d7-4c5f-bca1-f410b9341dc3 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:48.281771 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fe1563-a1d7-4c5f-bca1-f410b9341dc3 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:48.284185 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 386/762] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:15:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:15:48.288672 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16721604-5972-40b9-99c7-68fe5a91f4c9 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:48.289263 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16721604-5972-40b9-99c7-68fe5a91f4c9 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:48.289769 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16721604-5972-40b9-99c7-68fe5a91f4c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:48.307968 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-16721604-5972-40b9-99c7-68fe5a91f4c9 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:15:48.310050 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-16721604-5972-40b9-99c7-68fe5a91f4c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:15:48.332890 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9dc0c7-fe1e-4153-85b6-763838f6ba57 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:48.342926 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:48.387147 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 387/763] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:15:48 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 02:15:48.391513 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c73072-67de-4f09-91f8-64b684db0dc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:48.392572 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c73072-67de-4f09-91f8-64b684db0dc7 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:48.393080 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c73072-67de-4f09-91f8-64b684db0dc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:48.393554 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c73072-67de-4f09-91f8-64b684db0dc7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:48.405971 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c73072-67de-4f09-91f8-64b684db0dc7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:48.407539 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c73072-67de-4f09-91f8-64b684db0dc7 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:48.418859 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16721604-5972-40b9-99c7-68fe5a91f4c9 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:48.437479 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-62c73072-67de-4f09-91f8-64b684db0dc7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:15:48.437479 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:48.437479 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:48.437479 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:48.437479 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:48.437479 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:48.437479 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:48.437479 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:48.437479 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:48.437479 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:48.437479 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:48.437479 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:48.437479 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 02:15:48.437479 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 02:15:48.437479 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 02:15:48.438729 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 02:15:48.438729 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:48.438729 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:48.438729 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:48.438729 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:48.438729 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:48.438729 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:48.438729 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:48.438729 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:48.438729 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:48.438729 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:48.438729 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:48.438729 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:48.438729 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:48.438729 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:48.438729 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:48.438729 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:48.439670 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 02:15:48.439670 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 02:15:48.439670 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:15:48.439670 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:15:48.439670 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 377/764] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 18 02:15:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 02:15:48.451006 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c73072-67de-4f09-91f8-64b684db0dc7 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:48.572942 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:48.617884 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16721604-5972-40b9-99c7-68fe5a91f4c9 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:48.648910 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c73072-67de-4f09-91f8-64b684db0dc7 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:48.689667 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df24b1d-c649-4ec9-b4a4-02f0064ea7ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:48.690645 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df24b1d-c649-4ec9-b4a4-02f0064ea7ab None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:48.690929 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df24b1d-c649-4ec9-b4a4-02f0064ea7ab None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:48.691201 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df24b1d-c649-4ec9-b4a4-02f0064ea7ab None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:48.691861 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df24b1d-c649-4ec9-b4a4-02f0064ea7ab None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:48.693241 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df24b1d-c649-4ec9-b4a4-02f0064ea7ab None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:48.695213 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4973c59-85cc-4ee2-bed8-b38acdd57729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:48.696239 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4973c59-85cc-4ee2-bed8-b38acdd57729 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:48.696759 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4973c59-85cc-4ee2-bed8-b38acdd57729 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:48.697175 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4973c59-85cc-4ee2-bed8-b38acdd57729 None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:48.697915 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4973c59-85cc-4ee2-bed8-b38acdd57729 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:48.698270 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4973c59-85cc-4ee2-bed8-b38acdd57729 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:48.700185 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4973c59-85cc-4ee2-bed8-b38acdd57729 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:48.706865 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-6df24b1d-c649-4ec9-b4a4-02f0064ea7ab None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 18 02:15:48.706865 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:48.706865 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 02:15:48.706865 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 02:15:48.706865 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 02:15:48.706865 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 02:15:48.706865 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 02:15:48.706865 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 02:15:48.706865 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 18 02:15:48.706865 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:15:48.706865 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 02:15:48.706865 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:15:48.706865 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:48.706865 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:48.706865 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:48.706865 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:48.708017 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:48.708017 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:48.708017 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:48.708017 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:48.708017 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:48.708017 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:48.708017 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:48.708017 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 02:15:48.708017 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 02:15:48.708017 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 02:15:48.708017 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 02:15:48.708017 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:48.708017 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:48.708017 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:48.708017 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:48.708017 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:48.709078 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:48.709078 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:48.709078 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:48.709078 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:48.709078 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:48.709078 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:48.709078 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:48.709078 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:48.709078 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:48.709078 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:48.709078 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:48.709078 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 02:15:48.709078 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 02:15:48.709078 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 18 02:15:48.709078 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:15:48.718122 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 388/765] 10.222.0.66 () {56 vars in 1129 bytes} [Mon Jul 18 02:15:48 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 02:15:48.718361 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 378/766] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 18 02:15:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 02:15:48.720146 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df24b1d-c649-4ec9-b4a4-02f0064ea7ab None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:48.723241 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4973c59-85cc-4ee2-bed8-b38acdd57729 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:48.781041 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:48.896146 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4973c59-85cc-4ee2-bed8-b38acdd57729 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:48.900072 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df24b1d-c649-4ec9-b4a4-02f0064ea7ab None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:48.936315 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:48.953237 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3a3013-5ee4-45ec-a502-3ba17914b36b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:48.954039 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3a3013-5ee4-45ec-a502-3ba17914b36b None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:48.954353 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3a3013-5ee4-45ec-a502-3ba17914b36b None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:48.954625 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3a3013-5ee4-45ec-a502-3ba17914b36b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:48.955891 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45cb913-7554-4660-a18f-c98a0839c7a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:48.956582 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45cb913-7554-4660-a18f-c98a0839c7a2 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:48.956868 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45cb913-7554-4660-a18f-c98a0839c7a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:48.957146 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45cb913-7554-4660-a18f-c98a0839c7a2 None admin] PATH_INFO: `/v3/services` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:48.957869 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45cb913-7554-4660-a18f-c98a0839c7a2 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:48.960678 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45cb913-7554-4660-a18f-c98a0839c7a2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:48.960678 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45cb913-7554-4660-a18f-c98a0839c7a2 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:48.968857 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 389/767] 10.222.0.66 () {56 vars in 1124 bytes} [Mon Jul 18 02:15:48 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:48.970219 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3a3013-5ee4-45ec-a502-3ba17914b36b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:48.971879 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3a3013-5ee4-45ec-a502-3ba17914b36b None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:48.979048 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45cb913-7554-4660-a18f-c98a0839c7a2 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:48.992477 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-3c3a3013-5ee4-45ec-a502-3ba17914b36b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 02:15:48.992477 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:48.992477 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:48.992477 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:48.992477 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:48.992477 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:48.992477 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:48.992477 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:48.992477 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:48.992477 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:48.992477 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:48.992477 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:48.992477 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 02:15:48.992477 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 02:15:48.992477 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 02:15:48.993435 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 02:15:48.993435 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:48.993435 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:48.993435 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:48.993435 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:48.993435 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:48.993435 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:48.993435 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:48.993435 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:48.993435 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:48.993435 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:48.993435 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:48.993435 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:48.993435 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:48.993435 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:48.993435 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:48.993435 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:48.994379 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 02:15:48.994379 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 02:15:48.994379 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 02:15:48.994379 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 02:15:48.994379 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 02:15:48.994379 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 02:15:48.994379 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 02:15:48.994379 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:15:48.994379 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 379/768] 10.222.0.66 () {56 vars in 1111 bytes} [Mon Jul 18 02:15:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:49.006106 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3a3013-5ee4-45ec-a502-3ba17914b36b None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:49.116802 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3a3013-5ee4-45ec-a502-3ba17914b36b None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:49.157056 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cf911c-d4fc-4654-bc27-4b79271a72dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:49.157732 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cf911c-d4fc-4654-bc27-4b79271a72dc None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:49.158042 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cf911c-d4fc-4654-bc27-4b79271a72dc None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:49.158606 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cf911c-d4fc-4654-bc27-4b79271a72dc None admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:49.159403 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cf911c-d4fc-4654-bc27-4b79271a72dc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:49.159716 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cf911c-d4fc-4654-bc27-4b79271a72dc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:49.166121 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cf911c-d4fc-4654-bc27-4b79271a72dc None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:49.176380 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 380/769] 10.222.0.66 () {56 vars in 1148 bytes} [Mon Jul 18 02:15:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:49.181830 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cf911c-d4fc-4654-bc27-4b79271a72dc None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:49.231132 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:49.243928 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:49.264203 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45cb913-7554-4660-a18f-c98a0839c7a2 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:49.301462 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cf911c-d4fc-4654-bc27-4b79271a72dc None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:49.341343 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed688c50-4c0b-449e-a956-48d50149e62a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:49.342483 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed688c50-4c0b-449e-a956-48d50149e62a None admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:49.342815 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed688c50-4c0b-449e-a956-48d50149e62a None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:49.343162 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed688c50-4c0b-449e-a956-48d50149e62a None admin] PATH_INFO: `/v3/projects/76f7c0ae36ae49b09dcae4c3e9a875f4/users/80b138831bc54c7b986d50092d62d4af/roles/d4aa3324f0c84887b41d0dc4acf7f30e` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:49.350123 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed688c50-4c0b-449e-a956-48d50149e62a None admin] RBAC: Authorizing `identity:create_grant(project_id=76f7c0ae36ae49b09dcae4c3e9a875f4, user_id=80b138831bc54c7b986d50092d62d4af, role_id=d4aa3324f0c84887b41d0dc4acf7f30e)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:49.350123 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed688c50-4c0b-449e-a956-48d50149e62a None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:49.358881 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37063b41-c396-48f4-b79a-76bcdd0835ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:49.359623 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37063b41-c396-48f4-b79a-76bcdd0835ac None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:49.359915 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37063b41-c396-48f4-b79a-76bcdd0835ac None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:49.360196 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37063b41-c396-48f4-b79a-76bcdd0835ac None admin] PATH_INFO: `/v3/services` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:49.360959 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37063b41-c396-48f4-b79a-76bcdd0835ac None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:49.361324 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37063b41-c396-48f4-b79a-76bcdd0835ac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:49.362787 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37063b41-c396-48f4-b79a-76bcdd0835ac None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:49.368735 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 390/770] 10.222.0.66 () {56 vars in 1124 bytes} [Mon Jul 18 02:15:48 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 390 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:49.374660 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37063b41-c396-48f4-b79a-76bcdd0835ac None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:49.488018 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37063b41-c396-48f4-b79a-76bcdd0835ac None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:49.490018 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 381/771] 10.222.0.66 () {58 vars in 1439 bytes} [Mon Jul 18 02:15:49 2022] PUT /identity/v3/projects/76f7c0ae36ae49b09dcae4c3e9a875f4/users/80b138831bc54c7b986d50092d62d4af/roles/d4aa3324f0c84887b41d0dc4acf7f30e => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 02:15:49.519275 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb97485d-2b34-43fb-94f5-175a40e343bd None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:49.520147 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb97485d-2b34-43fb-94f5-175a40e343bd None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:49.520641 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb97485d-2b34-43fb-94f5-175a40e343bd None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:49.526068 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 382/772] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:15:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:15:49.527862 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59808535-5aa7-4314-99d2-e62de37223f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:49.528604 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59808535-5aa7-4314-99d2-e62de37223f7 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:49.528890 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59808535-5aa7-4314-99d2-e62de37223f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:49.529210 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59808535-5aa7-4314-99d2-e62de37223f7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:49.529919 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bdb8b4-5eaa-40d4-96b1-980c2689af03 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:49.530031 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59808535-5aa7-4314-99d2-e62de37223f7 None admin] RBAC: Adding query filter params (interface=public, service_id=81248ad034914996889def74f982aaa9, region_id=RegionOne) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:49.530369 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59808535-5aa7-4314-99d2-e62de37223f7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:49.530803 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bdb8b4-5eaa-40d4-96b1-980c2689af03 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:49.531296 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bdb8b4-5eaa-40d4-96b1-980c2689af03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:49.531854 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59808535-5aa7-4314-99d2-e62de37223f7 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:49.546151 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 391/773] 10.222.0.66 () {56 vars in 1251 bytes} [Mon Jul 18 02:15:49 2022] GET /identity/v3/endpoints?service_id=81248ad034914996889def74f982aaa9&interface=public®ion_id=RegionOne => generated 194 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:49.549787 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59808535-5aa7-4314-99d2-e62de37223f7 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:49.555538 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-b2bdb8b4-5eaa-40d4-96b1-980c2689af03 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:15:49.556855 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-b2bdb8b4-5eaa-40d4-96b1-980c2689af03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:15:49.585286 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:49.637179 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 383/774] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:15:49 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 02:15:49.663081 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bdb8b4-5eaa-40d4-96b1-980c2689af03 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:49.733341 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:49.736312 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59808535-5aa7-4314-99d2-e62de37223f7 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:49.779280 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d62662-b04a-4d2e-af15-a2260cef1f5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:49.779876 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d62662-b04a-4d2e-af15-a2260cef1f5d None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:49.780164 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d62662-b04a-4d2e-af15-a2260cef1f5d None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:49.780445 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d62662-b04a-4d2e-af15-a2260cef1f5d None admin] PATH_INFO: `/v3/services` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:49.781131 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d62662-b04a-4d2e-af15-a2260cef1f5d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:49.783252 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d62662-b04a-4d2e-af15-a2260cef1f5d None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:49.794111 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 392/775] 10.222.0.66 () {56 vars in 1087 bytes} [Mon Jul 18 02:15:49 2022] GET /identity/v3/services => generated 772 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 02:15:49.826771 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bdb8b4-5eaa-40d4-96b1-980c2689af03 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:49.865299 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3f5738-a881-4269-a01a-45be2a4952e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:49.866191 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3f5738-a881-4269-a01a-45be2a4952e1 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:49.866677 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3f5738-a881-4269-a01a-45be2a4952e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:49.866956 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3f5738-a881-4269-a01a-45be2a4952e1 None admin] PATH_INFO: `/v3/services/network` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:49.867620 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3f5738-a881-4269-a01a-45be2a4952e1 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:49.869005 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3f5738-a881-4269-a01a-45be2a4952e1 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:49.882137 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-af3f5738-a881-4269-a01a-45be2a4952e1 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 18 02:15:49.882137 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:49.882137 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 02:15:49.882137 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 02:15:49.882137 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 02:15:49.882137 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 02:15:49.882137 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 02:15:49.882137 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 02:15:49.882137 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 18 02:15:49.882137 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:15:49.882137 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 02:15:49.882137 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:15:49.882137 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:49.882137 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:49.882137 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:49.882137 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:49.882137 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:49.883210 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:49.883210 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:49.883210 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:49.883210 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:49.883210 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:49.883210 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:49.883210 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 02:15:49.883210 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 02:15:49.883210 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 02:15:49.883210 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 02:15:49.883210 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:49.883210 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:49.883210 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:49.883210 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:49.883210 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:49.883210 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:49.884618 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:49.884618 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:49.884618 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:49.884618 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:49.884618 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:49.884618 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:49.884618 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:49.884618 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:49.884618 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:49.884618 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:49.884618 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 02:15:49.884618 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 02:15:49.884618 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 18 02:15:49.884618 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:15:49.884618 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 384/776] 10.222.0.66 () {56 vars in 1111 bytes} [Mon Jul 18 02:15:49 2022] GET /identity/v3/services/network => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:49.888382 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcde547-fd01-4fbc-bf91-66821d341c6e None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:49.888672 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcde547-fd01-4fbc-bf91-66821d341c6e None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:49.888958 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcde547-fd01-4fbc-bf91-66821d341c6e None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:49.890457 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 393/777] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:15:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:15:49.895361 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3f5738-a881-4269-a01a-45be2a4952e1 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:49.897580 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43872b5-617e-48ea-8474-4c2f26abff6e None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:49.897864 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43872b5-617e-48ea-8474-4c2f26abff6e None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:49.898178 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43872b5-617e-48ea-8474-4c2f26abff6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:49.942147 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-e43872b5-617e-48ea-8474-4c2f26abff6e None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:15:49.943534 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-e43872b5-617e-48ea-8474-4c2f26abff6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:15:50.004849 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:50.010988 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3f5738-a881-4269-a01a-45be2a4952e1 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:50.057795 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3d8543-bafd-47eb-a555-fb8f8f7875dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:50.059053 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 394/778] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:15:49 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 364 bytes (3 switches on core 0) Jul 18 02:15:50.059660 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3d8543-bafd-47eb-a555-fb8f8f7875dc None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:50.060905 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3d8543-bafd-47eb-a555-fb8f8f7875dc None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:50.061395 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3d8543-bafd-47eb-a555-fb8f8f7875dc None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:50.062421 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3d8543-bafd-47eb-a555-fb8f8f7875dc None admin] RBAC: Adding query filter params (name=network) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:50.063104 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3d8543-bafd-47eb-a555-fb8f8f7875dc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:50.064733 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3d8543-bafd-47eb-a555-fb8f8f7875dc None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:50.074219 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 385/779] 10.222.0.66 () {56 vars in 1112 bytes} [Mon Jul 18 02:15:49 2022] GET /identity/v3/services?name=network => generated 124 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:50.076620 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43872b5-617e-48ea-8474-4c2f26abff6e None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:50.087092 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3d8543-bafd-47eb-a555-fb8f8f7875dc None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:50.100267 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:50.196383 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3d8543-bafd-47eb-a555-fb8f8f7875dc None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:50.201232 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43872b5-617e-48ea-8474-4c2f26abff6e None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:50.234013 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e71484-169a-40ea-b856-9a52323cb57e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:50.235447 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e71484-169a-40ea-b856-9a52323cb57e None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:50.235923 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e71484-169a-40ea-b856-9a52323cb57e None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:50.236374 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e71484-169a-40ea-b856-9a52323cb57e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:50.239928 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680842bf-1516-4bc9-b5ca-7fa1fb3fd8b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:50.240848 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680842bf-1516-4bc9-b5ca-7fa1fb3fd8b2 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:50.241350 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680842bf-1516-4bc9-b5ca-7fa1fb3fd8b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:50.241878 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680842bf-1516-4bc9-b5ca-7fa1fb3fd8b2 None admin] PATH_INFO: `/v3/services` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:50.243091 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680842bf-1516-4bc9-b5ca-7fa1fb3fd8b2 None admin] RBAC: Adding query filter params (type=network) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:50.243649 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680842bf-1516-4bc9-b5ca-7fa1fb3fd8b2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:50.245243 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680842bf-1516-4bc9-b5ca-7fa1fb3fd8b2 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:50.247567 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e71484-169a-40ea-b856-9a52323cb57e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 02:15:50.255496 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-c7e71484-169a-40ea-b856-9a52323cb57e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 02:15:50.255496 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:50.255496 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:50.255496 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:50.255496 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:50.255496 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:50.255496 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:50.255496 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:50.255496 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:50.255496 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:50.255496 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:50.255496 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:50.255496 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 02:15:50.255496 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 02:15:50.255496 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 02:15:50.255496 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 02:15:50.256483 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 02:15:50.256483 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 02:15:50.256483 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 02:15:50.256483 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 02:15:50.256483 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:50.256483 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:50.256483 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:50.256483 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:50.256483 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:50.256483 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:50.256483 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:50.256483 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:50.256483 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:50.256483 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:50.256483 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:50.256483 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:50.257949 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:50.257949 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:50.257949 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:50.257949 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:50.257949 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 02:15:50.257949 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 02:15:50.257949 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 02:15:50.257949 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 02:15:50.257949 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 02:15:50.257949 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 02:15:50.257949 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 02:15:50.257949 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:15:50.257949 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 386/781] 10.222.0.66 () {56 vars in 1102 bytes} [Mon Jul 18 02:15:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 02:15:50.259207 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 395/781] 10.222.0.66 () {56 vars in 1112 bytes} [Mon Jul 18 02:15:50 2022] GET /identity/v3/services?type=network => generated 350 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:50.264010 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680842bf-1516-4bc9-b5ca-7fa1fb3fd8b2 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:50.264609 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e71484-169a-40ea-b856-9a52323cb57e None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:50.292277 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:50.358189 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:50.380179 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680842bf-1516-4bc9-b5ca-7fa1fb3fd8b2 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:50.422741 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d98ed6e-3641-4c93-9608-edfe0aa67275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:50.423470 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d98ed6e-3641-4c93-9608-edfe0aa67275 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:50.423703 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d98ed6e-3641-4c93-9608-edfe0aa67275 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:50.430194 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d98ed6e-3641-4c93-9608-edfe0aa67275 None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:50.430194 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d98ed6e-3641-4c93-9608-edfe0aa67275 None admin] RBAC: Adding query filter params (name=service) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:50.430194 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d98ed6e-3641-4c93-9608-edfe0aa67275 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:50.430194 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d98ed6e-3641-4c93-9608-edfe0aa67275 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:50.436075 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e71484-169a-40ea-b856-9a52323cb57e None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:50.440979 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 396/782] 10.222.0.66 () {56 vars in 1103 bytes} [Mon Jul 18 02:15:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:50.446108 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d98ed6e-3641-4c93-9608-edfe0aa67275 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:50.509480 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d85706-2ff8-4117-aac3-d28677a2bc29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:50.510174 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d85706-2ff8-4117-aac3-d28677a2bc29 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:50.510465 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d85706-2ff8-4117-aac3-d28677a2bc29 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:50.510747 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d85706-2ff8-4117-aac3-d28677a2bc29 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:50.512172 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d85706-2ff8-4117-aac3-d28677a2bc29 None admin] RBAC: Adding query filter params (interface=public, service_id=1131243c36c742fd87d542372646ecb3, region_id=RegionOne) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:50.512491 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d85706-2ff8-4117-aac3-d28677a2bc29 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:50.519409 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d85706-2ff8-4117-aac3-d28677a2bc29 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:50.534050 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 387/783] 10.222.0.66 () {56 vars in 1251 bytes} [Mon Jul 18 02:15:50 2022] GET /identity/v3/endpoints?service_id=1131243c36c742fd87d542372646ecb3&interface=public®ion_id=RegionOne => generated 194 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:50.547665 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d85706-2ff8-4117-aac3-d28677a2bc29 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:50.595475 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d98ed6e-3641-4c93-9608-edfe0aa67275 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:50.632442 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2758d367-d280-4890-83ae-a1b99222300a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:50.633240 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2758d367-d280-4890-83ae-a1b99222300a None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:50.633526 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2758d367-d280-4890-83ae-a1b99222300a None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:50.633801 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2758d367-d280-4890-83ae-a1b99222300a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:50.645944 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2758d367-d280-4890-83ae-a1b99222300a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:50.647693 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2758d367-d280-4890-83ae-a1b99222300a None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:50.649432 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:50.666718 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-2758d367-d280-4890-83ae-a1b99222300a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:15:50.666718 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:50.666718 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:50.666718 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:50.666718 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:50.666718 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:50.666718 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:50.666718 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:50.666718 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:50.666718 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:50.666718 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:50.666718 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:50.666718 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 02:15:50.666718 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 02:15:50.666718 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 02:15:50.667798 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 02:15:50.667798 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:50.667798 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:50.667798 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:50.667798 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:50.667798 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:50.667798 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:50.667798 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:50.667798 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:50.667798 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:50.667798 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:50.667798 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:50.667798 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:50.667798 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:50.667798 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:50.667798 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:50.667798 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:50.668719 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 02:15:50.668719 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 02:15:50.668719 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:15:50.668719 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:15:50.668719 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 397/784] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 18 02:15:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:50.674032 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c2f97b-b49c-4e87-a441-bb9ac001d61b None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:50.674032 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c2f97b-b49c-4e87-a441-bb9ac001d61b None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:50.674032 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c2f97b-b49c-4e87-a441-bb9ac001d61b None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:50.674032 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 398/785] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:15:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:15:50.674032 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c2f97b-b49c-4e87-a441-bb9ac001d61b None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:50.693695 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:50.710489 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d85706-2ff8-4117-aac3-d28677a2bc29 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:50.748391 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc1b9c1-e672-45ba-9d23-87f93c8cbb46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:50.749293 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc1b9c1-e672-45ba-9d23-87f93c8cbb46 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:50.749650 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc1b9c1-e672-45ba-9d23-87f93c8cbb46 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:50.750178 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc1b9c1-e672-45ba-9d23-87f93c8cbb46 None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:50.751014 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc1b9c1-e672-45ba-9d23-87f93c8cbb46 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:50.752479 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc1b9c1-e672-45ba-9d23-87f93c8cbb46 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:50.758385 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 388/786] 10.222.0.66 () {56 vars in 1087 bytes} [Mon Jul 18 02:15:50 2022] GET /identity/v3/services => generated 772 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:50.759880 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df7c4cd-0edb-4c93-a065-7341ebe87fc0 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:50.760236 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df7c4cd-0edb-4c93-a065-7341ebe87fc0 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:50.760581 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df7c4cd-0edb-4c93-a065-7341ebe87fc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:50.780487 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-5df7c4cd-0edb-4c93-a065-7341ebe87fc0 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:15:50.782068 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-5df7c4cd-0edb-4c93-a065-7341ebe87fc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:15:50.788970 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c2f97b-b49c-4e87-a441-bb9ac001d61b None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:50.827993 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970bbe3f-7827-40ce-a31b-2b1f68c33649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:50.828777 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970bbe3f-7827-40ce-a31b-2b1f68c33649 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:50.829068 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970bbe3f-7827-40ce-a31b-2b1f68c33649 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:50.829350 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970bbe3f-7827-40ce-a31b-2b1f68c33649 None admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:50.830128 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970bbe3f-7827-40ce-a31b-2b1f68c33649 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:50.831776 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970bbe3f-7827-40ce-a31b-2b1f68c33649 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:50.833321 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970bbe3f-7827-40ce-a31b-2b1f68c33649 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:50.848030 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 399/787] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 18 02:15:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:50.858843 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970bbe3f-7827-40ce-a31b-2b1f68c33649 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:50.926420 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 389/788] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:15:50 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 02:15:50.951477 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df7c4cd-0edb-4c93-a065-7341ebe87fc0 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:50.975222 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:51.028631 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970bbe3f-7827-40ce-a31b-2b1f68c33649 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:51.065464 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:51.069178 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87741521-b711-48e9-b031-b42bf605fcb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:51.070268 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87741521-b711-48e9-b031-b42bf605fcb7 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:51.070721 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87741521-b711-48e9-b031-b42bf605fcb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:51.071185 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87741521-b711-48e9-b031-b42bf605fcb7 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:51.079817 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87741521-b711-48e9-b031-b42bf605fcb7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:51.081700 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87741521-b711-48e9-b031-b42bf605fcb7 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:51.093565 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df7c4cd-0edb-4c93-a065-7341ebe87fc0 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:51.103084 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-87741521-b711-48e9-b031-b42bf605fcb7 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 02:15:51.103084 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:51.103084 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:51.103084 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:51.103084 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:51.103084 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:51.103084 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:51.103084 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:51.103084 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:51.103084 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:51.103084 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:51.103084 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:51.103084 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 02:15:51.103084 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 02:15:51.103084 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 02:15:51.103084 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 02:15:51.104052 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:51.104052 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:51.104052 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 02:15:51.104052 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 02:15:51.104052 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 02:15:51.104052 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 02:15:51.104052 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:51.104052 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:51.104052 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:51.104052 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:51.104052 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:51.104052 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:51.104052 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:51.104052 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:51.104052 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:51.104052 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:51.104052 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:51.104052 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:51.105564 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:51.105564 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:51.105564 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 02:15:51.105564 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 02:15:51.105564 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 02:15:51.105564 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 02:15:51.105564 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 02:15:51.105564 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 02:15:51.105564 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 02:15:51.105564 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:15:51.108238 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 400/789] 10.222.0.66 () {56 vars in 1117 bytes} [Mon Jul 18 02:15:50 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:51.114692 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87741521-b711-48e9-b031-b42bf605fcb7 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:51.140095 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d0caff-ee66-4f76-ad18-c3d1493944ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:51.142026 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d0caff-ee66-4f76-ad18-c3d1493944ff None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:51.143704 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d0caff-ee66-4f76-ad18-c3d1493944ff None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:51.144963 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d0caff-ee66-4f76-ad18-c3d1493944ff None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:51.164072 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d0caff-ee66-4f76-ad18-c3d1493944ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 02:15:51.180657 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-d5d0caff-ee66-4f76-ad18-c3d1493944ff None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 02:15:51.180657 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:51.180657 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:51.180657 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:51.180657 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:51.180657 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:51.180657 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:51.180657 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:51.180657 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:51.180657 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:51.180657 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:51.180657 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:51.180657 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 02:15:51.180657 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 02:15:51.180657 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 02:15:51.180657 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 02:15:51.181734 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 02:15:51.181734 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 02:15:51.181734 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 02:15:51.181734 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 02:15:51.181734 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:51.181734 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:51.181734 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:51.181734 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:51.181734 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:51.181734 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:51.181734 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:51.181734 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:51.181734 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:51.181734 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:51.181734 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:51.181734 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:51.182646 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:51.182646 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:51.182646 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:51.182646 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:51.182646 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 02:15:51.182646 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 02:15:51.182646 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 02:15:51.182646 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 02:15:51.182646 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 02:15:51.182646 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 02:15:51.182646 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 02:15:51.182646 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:15:51.190269 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 390/790] 10.222.0.66 () {56 vars in 1096 bytes} [Mon Jul 18 02:15:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 02:15:51.196093 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d0caff-ee66-4f76-ad18-c3d1493944ff None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:51.301542 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87741521-b711-48e9-b031-b42bf605fcb7 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:51.322800 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:51.353746 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604e6c03-1b1f-472f-b466-005f6e0454ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:51.355054 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604e6c03-1b1f-472f-b466-005f6e0454ff None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:51.357236 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604e6c03-1b1f-472f-b466-005f6e0454ff None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:51.357732 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604e6c03-1b1f-472f-b466-005f6e0454ff None admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:51.358874 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604e6c03-1b1f-472f-b466-005f6e0454ff None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:51.361193 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604e6c03-1b1f-472f-b466-005f6e0454ff None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:51.364580 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604e6c03-1b1f-472f-b466-005f6e0454ff None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:51.393039 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 401/791] 10.222.0.66 () {56 vars in 1149 bytes} [Mon Jul 18 02:15:51 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:51.399555 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604e6c03-1b1f-472f-b466-005f6e0454ff None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:51.413036 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d0caff-ee66-4f76-ad18-c3d1493944ff None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:51.492322 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dfd386-4662-4670-a0b2-1dbb934cc7ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:51.493981 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dfd386-4662-4670-a0b2-1dbb934cc7ee None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:51.496073 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dfd386-4662-4670-a0b2-1dbb934cc7ee None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:51.496577 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dfd386-4662-4670-a0b2-1dbb934cc7ee None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:51.497535 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dfd386-4662-4670-a0b2-1dbb934cc7ee None admin] RBAC: Adding query filter params (name=admin) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:51.498781 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dfd386-4662-4670-a0b2-1dbb934cc7ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:51.501203 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dfd386-4662-4670-a0b2-1dbb934cc7ee None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:51.515236 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:51.525610 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 391/792] 10.222.0.66 () {56 vars in 1099 bytes} [Mon Jul 18 02:15:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 02:15:51.527309 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dfd386-4662-4670-a0b2-1dbb934cc7ee None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:51.596053 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604e6c03-1b1f-472f-b466-005f6e0454ff None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:51.654440 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0335ba-705d-4e8e-a644-c469606d1a9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:51.655246 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0335ba-705d-4e8e-a644-c469606d1a9e None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:51.655533 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0335ba-705d-4e8e-a644-c469606d1a9e None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:51.655809 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0335ba-705d-4e8e-a644-c469606d1a9e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:51.677057 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0335ba-705d-4e8e-a644-c469606d1a9e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:51.681113 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0335ba-705d-4e8e-a644-c469606d1a9e None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:51.705178 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dfd386-4662-4670-a0b2-1dbb934cc7ee None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:51.722425 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-dd0335ba-705d-4e8e-a644-c469606d1a9e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:15:51.722425 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:51.722425 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:51.722425 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:51.722425 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:51.722425 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:51.722425 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:51.722425 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:51.722425 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:51.722425 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:51.722425 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:51.722425 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:51.722425 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 02:15:51.722425 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 02:15:51.722425 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 02:15:51.723406 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 02:15:51.723406 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:51.723406 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:51.723406 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:51.723406 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:51.723406 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:51.723406 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:51.723406 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:51.723406 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:51.723406 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:51.723406 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:51.723406 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:51.723406 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:51.723406 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:51.723406 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:51.723406 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:51.723406 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:51.724319 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 02:15:51.724319 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 02:15:51.724319 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:15:51.724319 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:15:51.724319 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 402/793] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 18 02:15:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:51.730360 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0335ba-705d-4e8e-a644-c469606d1a9e None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:51.777865 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e34e3c8-6d84-409b-8261-e4f42ec2f69f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:51.778617 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e34e3c8-6d84-409b-8261-e4f42ec2f69f None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:51.779177 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e34e3c8-6d84-409b-8261-e4f42ec2f69f None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:51.779460 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e34e3c8-6d84-409b-8261-e4f42ec2f69f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:51.779905 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:51.803994 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e34e3c8-6d84-409b-8261-e4f42ec2f69f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:51.805569 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e34e3c8-6d84-409b-8261-e4f42ec2f69f None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:51.836678 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-6e34e3c8-6d84-409b-8261-e4f42ec2f69f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:15:51.836678 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:51.836678 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:51.836678 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:51.836678 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:51.836678 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:51.836678 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:51.836678 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:51.836678 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:51.836678 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:51.836678 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:51.836678 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:51.836678 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 02:15:51.836678 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 02:15:51.836678 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 02:15:51.837652 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 02:15:51.837652 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:51.837652 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:51.837652 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:51.837652 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:51.837652 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:51.837652 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:51.837652 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:51.837652 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:51.837652 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:51.837652 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:51.837652 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:51.837652 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:51.837652 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:51.837652 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:51.837652 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:51.837652 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:51.838608 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 02:15:51.838608 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 02:15:51.838608 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:15:51.838608 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:15:51.845224 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 392/794] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 18 02:15:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:51.851125 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e34e3c8-6d84-409b-8261-e4f42ec2f69f None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:51.924791 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0335ba-705d-4e8e-a644-c469606d1a9e None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:51.958591 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:51.999904 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b215fbb-3b59-4738-b131-938968eae29a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:52.000636 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b215fbb-3b59-4738-b131-938968eae29a None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:52.000929 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b215fbb-3b59-4738-b131-938968eae29a None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:52.001208 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b215fbb-3b59-4738-b131-938968eae29a None admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:52.001943 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b215fbb-3b59-4738-b131-938968eae29a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:52.002288 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b215fbb-3b59-4738-b131-938968eae29a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:52.003681 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b215fbb-3b59-4738-b131-938968eae29a None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:52.019296 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 403/795] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 18 02:15:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:52.021283 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9475896c-aa68-4702-b2f6-378a7b51cc7a None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:52.021587 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9475896c-aa68-4702-b2f6-378a7b51cc7a None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:52.021870 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9475896c-aa68-4702-b2f6-378a7b51cc7a None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:52.026188 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 404/796] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:15:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:15:52.027668 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9475896c-aa68-4702-b2f6-378a7b51cc7a None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:52.041434 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e34e3c8-6d84-409b-8261-e4f42ec2f69f None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:52.089754 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5791f60d-89d9-4318-b0c0-e9d42cf542f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:52.090464 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5791f60d-89d9-4318-b0c0-e9d42cf542f4 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:52.090753 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5791f60d-89d9-4318-b0c0-e9d42cf542f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:52.091039 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5791f60d-89d9-4318-b0c0-e9d42cf542f4 None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:52.092292 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5791f60d-89d9-4318-b0c0-e9d42cf542f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:52.092615 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5791f60d-89d9-4318-b0c0-e9d42cf542f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:52.094323 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5791f60d-89d9-4318-b0c0-e9d42cf542f4 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:52.112941 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 393/797] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 18 02:15:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:52.114370 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab16886-4677-40bf-8f4e-3aa907797c84 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:52.114664 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab16886-4677-40bf-8f4e-3aa907797c84 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:52.114944 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab16886-4677-40bf-8f4e-3aa907797c84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:52.131920 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:52.135437 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-cab16886-4677-40bf-8f4e-3aa907797c84 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:15:52.137093 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-cab16886-4677-40bf-8f4e-3aa907797c84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:15:52.155022 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9475896c-aa68-4702-b2f6-378a7b51cc7a None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:52.200212 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8c4cb0-fbc0-4fa6-82c2-15cb13a0a129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:52.201383 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8c4cb0-fbc0-4fa6-82c2-15cb13a0a129 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:52.201893 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8c4cb0-fbc0-4fa6-82c2-15cb13a0a129 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:52.202401 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8c4cb0-fbc0-4fa6-82c2-15cb13a0a129 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:52.209935 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8c4cb0-fbc0-4fa6-82c2-15cb13a0a129 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:52.211815 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8c4cb0-fbc0-4fa6-82c2-15cb13a0a129 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:52.224197 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 394/798] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:15:52 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 02:15:52.225315 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab16886-4677-40bf-8f4e-3aa907797c84 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:52.235953 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-ed8c4cb0-fbc0-4fa6-82c2-15cb13a0a129 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 02:15:52.235953 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:52.235953 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:52.235953 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:52.235953 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:52.235953 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:52.235953 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:52.235953 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:52.235953 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:52.235953 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:52.235953 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:52.235953 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:52.235953 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 02:15:52.235953 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 02:15:52.235953 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 02:15:52.236934 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 02:15:52.236934 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:52.236934 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:52.236934 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:52.236934 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:52.236934 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:52.236934 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:52.236934 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:52.236934 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:52.236934 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:52.236934 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:52.236934 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:52.236934 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:52.236934 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:52.236934 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:52.236934 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:52.236934 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:52.237839 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 02:15:52.237839 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 02:15:52.237839 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 02:15:52.237839 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 02:15:52.237839 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 02:15:52.237839 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 02:15:52.237839 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 02:15:52.237839 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:15:52.242333 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 405/799] 10.222.0.66 () {56 vars in 1111 bytes} [Mon Jul 18 02:15:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:52.250430 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8c4cb0-fbc0-4fa6-82c2-15cb13a0a129 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:52.294659 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:52.369423 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8c4cb0-fbc0-4fa6-82c2-15cb13a0a129 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:52.369828 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab16886-4677-40bf-8f4e-3aa907797c84 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:52.407328 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5d22d6-ac5f-42c5-aac2-fd444fb4cbed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:52.408134 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5d22d6-ac5f-42c5-aac2-fd444fb4cbed None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:52.408334 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c118f29-a662-47b1-8c3d-c1580a9f512e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:52.408540 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5d22d6-ac5f-42c5-aac2-fd444fb4cbed None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:52.408712 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5d22d6-ac5f-42c5-aac2-fd444fb4cbed None admin] PATH_INFO: `/v3/users/swift` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:52.408983 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c118f29-a662-47b1-8c3d-c1580a9f512e None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:52.409267 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c118f29-a662-47b1-8c3d-c1580a9f512e None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:52.409540 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c118f29-a662-47b1-8c3d-c1580a9f512e None admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:52.410348 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c118f29-a662-47b1-8c3d-c1580a9f512e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:52.410662 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c118f29-a662-47b1-8c3d-c1580a9f512e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:52.412097 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c118f29-a662-47b1-8c3d-c1580a9f512e None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:52.415953 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5d22d6-ac5f-42c5-aac2-fd444fb4cbed None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:52.417934 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5d22d6-ac5f-42c5-aac2-fd444fb4cbed None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:52.430114 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 406/800] 10.222.0.66 () {56 vars in 1148 bytes} [Mon Jul 18 02:15:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 02:15:52.430114 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c118f29-a662-47b1-8c3d-c1580a9f512e None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:52.433462 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-7f5d22d6-ac5f-42c5-aac2-fd444fb4cbed None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 18 02:15:52.433462 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:52.433462 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:52.433462 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:52.433462 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:52.433462 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:52.433462 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:52.433462 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:52.433462 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:52.433462 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:52.433462 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:52.433462 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:52.433462 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 02:15:52.433462 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 02:15:52.433462 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 02:15:52.433462 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 02:15:52.434430 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:52.434430 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:52.434430 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 02:15:52.434430 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 02:15:52.434430 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 02:15:52.434430 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 02:15:52.434430 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:52.434430 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:52.434430 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:52.434430 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:52.434430 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:52.434430 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:52.434430 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:52.434430 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:52.434430 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:52.434430 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:52.434430 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:52.434430 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:52.435371 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:52.435371 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:52.435371 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 02:15:52.435371 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 02:15:52.435371 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 02:15:52.435371 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 02:15:52.435371 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 02:15:52.435371 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 02:15:52.435371 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 18 02:15:52.435371 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:15:52.435371 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 395/801] 10.222.0.66 () {56 vars in 1096 bytes} [Mon Jul 18 02:15:52 2022] GET /identity/v3/users/swift => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:52.435970 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fed8e93-af72-4032-885b-970fda8fc47c None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:52.437324 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fed8e93-af72-4032-885b-970fda8fc47c None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:52.437324 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fed8e93-af72-4032-885b-970fda8fc47c None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:52.437847 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:52.438016 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 396/802] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:15:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:15:52.438816 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fed8e93-af72-4032-885b-970fda8fc47c None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:52.543944 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c118f29-a662-47b1-8c3d-c1580a9f512e None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:52.545018 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fed8e93-af72-4032-885b-970fda8fc47c None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:52.546798 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:52.582395 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c99329-9ed5-4bf4-a543-5001e68db506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:52.583278 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c99329-9ed5-4bf4-a543-5001e68db506 None admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:52.583415 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fb086d-6cba-40f2-a8c7-c6121868e7ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:52.583627 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c99329-9ed5-4bf4-a543-5001e68db506 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:52.583827 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c99329-9ed5-4bf4-a543-5001e68db506 None admin] PATH_INFO: `/v3/projects/76f7c0ae36ae49b09dcae4c3e9a875f4/users/768f872096584d4093fc616d547aabb6/roles/eea94daf3312421cb12aa8abfa73160e` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:52.584165 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fb086d-6cba-40f2-a8c7-c6121868e7ea None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:52.584449 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fb086d-6cba-40f2-a8c7-c6121868e7ea None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:52.584724 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fb086d-6cba-40f2-a8c7-c6121868e7ea None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:52.597479 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fb086d-6cba-40f2-a8c7-c6121868e7ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 02:15:52.599150 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c99329-9ed5-4bf4-a543-5001e68db506 None admin] RBAC: Authorizing `identity:create_grant(project_id=76f7c0ae36ae49b09dcae4c3e9a875f4, user_id=768f872096584d4093fc616d547aabb6, role_id=eea94daf3312421cb12aa8abfa73160e)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:52.600857 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c99329-9ed5-4bf4-a543-5001e68db506 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:52.605674 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-12fb086d-6cba-40f2-a8c7-c6121868e7ea None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 02:15:52.605674 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:52.605674 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:52.605674 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:52.605674 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:52.605674 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:52.605674 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:52.605674 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:52.605674 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:52.605674 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:52.605674 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:52.605674 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:52.605674 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 02:15:52.605674 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 02:15:52.605674 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 02:15:52.605674 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 02:15:52.606653 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 02:15:52.606653 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 02:15:52.606653 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 02:15:52.606653 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 02:15:52.606653 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:52.606653 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:52.606653 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:52.606653 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:52.606653 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:52.606653 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:52.606653 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:52.606653 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:52.606653 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:52.606653 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:52.606653 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:52.606653 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:52.611023 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:52.611023 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:52.611023 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:52.611023 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:52.611023 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 02:15:52.611023 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 02:15:52.611023 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 02:15:52.611023 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 02:15:52.611023 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 02:15:52.611023 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 02:15:52.611023 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 02:15:52.611023 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:15:52.611023 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 407/803] 10.222.0.66 () {56 vars in 1096 bytes} [Mon Jul 18 02:15:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:52.611023 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fb086d-6cba-40f2-a8c7-c6121868e7ea None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:52.675402 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 397/804] 10.222.0.66 () {58 vars in 1439 bytes} [Mon Jul 18 02:15:52 2022] PUT /identity/v3/projects/76f7c0ae36ae49b09dcae4c3e9a875f4/users/768f872096584d4093fc616d547aabb6/roles/eea94daf3312421cb12aa8abfa73160e => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:15:52.677448 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073d5ac8-d6c9-4e0a-a892-9c9fe2ff96a9 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:52.677745 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073d5ac8-d6c9-4e0a-a892-9c9fe2ff96a9 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:52.678024 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073d5ac8-d6c9-4e0a-a892-9c9fe2ff96a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:52.687717 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:52.699097 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-073d5ac8-d6c9-4e0a-a892-9c9fe2ff96a9 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:15:52.700406 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-073d5ac8-d6c9-4e0a-a892-9c9fe2ff96a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:15:52.734452 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fb086d-6cba-40f2-a8c7-c6121868e7ea None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:52.784504 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 398/805] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:15:52 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 02:15:52.785467 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073d5ac8-d6c9-4e0a-a892-9c9fe2ff96a9 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:52.788268 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023dba8c-387e-4299-a377-56b79a4593e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:52.789924 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023dba8c-387e-4299-a377-56b79a4593e4 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:52.790379 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023dba8c-387e-4299-a377-56b79a4593e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:52.790758 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023dba8c-387e-4299-a377-56b79a4593e4 None admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:52.791788 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023dba8c-387e-4299-a377-56b79a4593e4 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:52.792235 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023dba8c-387e-4299-a377-56b79a4593e4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:52.794275 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023dba8c-387e-4299-a377-56b79a4593e4 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:52.817836 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 408/806] 10.222.0.66 () {56 vars in 1135 bytes} [Mon Jul 18 02:15:52 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:52.818608 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023dba8c-387e-4299-a377-56b79a4593e4 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:52.886890 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:52.912986 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073d5ac8-d6c9-4e0a-a892-9c9fe2ff96a9 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:52.952401 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023dba8c-387e-4299-a377-56b79a4593e4 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:52.956069 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce30d4bd-5893-45f5-9db3-35e3c747f867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:52.956918 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce30d4bd-5893-45f5-9db3-35e3c747f867 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:52.957350 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce30d4bd-5893-45f5-9db3-35e3c747f867 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:52.957771 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce30d4bd-5893-45f5-9db3-35e3c747f867 None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:52.958701 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce30d4bd-5893-45f5-9db3-35e3c747f867 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:52.959173 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce30d4bd-5893-45f5-9db3-35e3c747f867 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:52.960879 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce30d4bd-5893-45f5-9db3-35e3c747f867 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:52.973328 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 399/807] 10.222.0.66 () {56 vars in 1099 bytes} [Mon Jul 18 02:15:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:52.974608 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce30d4bd-5893-45f5-9db3-35e3c747f867 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:52.977517 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:52.990143 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aad3184-c432-4e51-bcac-74a1b3da45e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:52.993571 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aad3184-c432-4e51-bcac-74a1b3da45e0 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:52.993571 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aad3184-c432-4e51-bcac-74a1b3da45e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:52.993571 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aad3184-c432-4e51-bcac-74a1b3da45e0 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:52.993571 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aad3184-c432-4e51-bcac-74a1b3da45e0 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:52.994376 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aad3184-c432-4e51-bcac-74a1b3da45e0 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:53.009183 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-5aad3184-c432-4e51-bcac-74a1b3da45e0 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 18 02:15:53.009183 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:53.009183 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 02:15:53.009183 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 02:15:53.009183 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 02:15:53.009183 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 02:15:53.009183 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 02:15:53.009183 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 02:15:53.009183 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 18 02:15:53.009183 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:15:53.009183 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 02:15:53.009183 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:15:53.009183 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:53.009183 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:53.009183 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:53.009183 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:53.010437 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:53.010437 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:53.010437 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:53.010437 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:53.010437 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:53.010437 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:53.010437 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:53.010437 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 02:15:53.010437 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 02:15:53.010437 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 02:15:53.010437 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 02:15:53.010437 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:53.010437 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:53.010437 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:53.010437 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:53.010437 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:53.011597 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:53.011597 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:53.011597 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:53.011597 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:53.011597 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:53.011597 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:53.011597 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:53.011597 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:53.011597 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:53.011597 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:53.011597 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:53.011597 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 02:15:53.011597 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 02:15:53.011597 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 18 02:15:53.011597 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:15:53.018752 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 409/808] 10.222.0.66 () {56 vars in 1129 bytes} [Mon Jul 18 02:15:52 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:53.020726 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aad3184-c432-4e51-bcac-74a1b3da45e0 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:53.091900 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce30d4bd-5893-45f5-9db3-35e3c747f867 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:53.126782 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1da5fc-b4cd-4b2a-b5de-e7ae090fde5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:53.127577 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1da5fc-b4cd-4b2a-b5de-e7ae090fde5e None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:53.127858 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1da5fc-b4cd-4b2a-b5de-e7ae090fde5e None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:53.128129 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1da5fc-b4cd-4b2a-b5de-e7ae090fde5e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:53.139502 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1da5fc-b4cd-4b2a-b5de-e7ae090fde5e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:53.140689 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aad3184-c432-4e51-bcac-74a1b3da45e0 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:53.140953 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1da5fc-b4cd-4b2a-b5de-e7ae090fde5e None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:53.144223 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:53.159636 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-fd1da5fc-b4cd-4b2a-b5de-e7ae090fde5e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:15:53.159636 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:53.159636 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:53.159636 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:53.159636 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:53.159636 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:53.159636 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:53.159636 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:53.159636 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:53.159636 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:53.159636 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:53.159636 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:53.159636 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 02:15:53.159636 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 02:15:53.159636 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 02:15:53.160569 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 02:15:53.160569 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:53.160569 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:53.160569 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:53.160569 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:53.160569 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:53.160569 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:53.160569 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:53.160569 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:53.160569 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:53.160569 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:53.160569 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:53.160569 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:53.160569 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:53.160569 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:53.160569 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:53.160569 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:53.161571 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 02:15:53.161571 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 02:15:53.161571 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:15:53.161571 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:15:53.161571 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 400/809] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 18 02:15:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:53.161863 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1da5fc-b4cd-4b2a-b5de-e7ae090fde5e None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:53.181701 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b379f2-5aff-4ad6-8a55-13ca4b8f1674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:53.182905 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b379f2-5aff-4ad6-8a55-13ca4b8f1674 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:53.183191 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b379f2-5aff-4ad6-8a55-13ca4b8f1674 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:53.183467 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b379f2-5aff-4ad6-8a55-13ca4b8f1674 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:53.194686 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b379f2-5aff-4ad6-8a55-13ca4b8f1674 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:53.196140 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b379f2-5aff-4ad6-8a55-13ca4b8f1674 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:53.214522 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-17b379f2-5aff-4ad6-8a55-13ca4b8f1674 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:15:53.214522 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:53.214522 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:53.214522 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:53.214522 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:53.214522 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:53.214522 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:53.214522 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:53.214522 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:53.214522 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:53.214522 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:53.214522 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:53.214522 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 02:15:53.214522 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 02:15:53.214522 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 02:15:53.215642 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 02:15:53.215642 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:53.215642 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:53.215642 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:53.215642 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:53.215642 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:53.215642 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:53.215642 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:53.215642 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:53.215642 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:53.215642 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:53.215642 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:53.215642 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:53.215642 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:53.215642 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:53.215642 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:53.215642 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:53.216584 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 02:15:53.216584 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 02:15:53.216584 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:15:53.216584 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:15:53.216584 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 410/810] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 18 02:15:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:53.217264 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6617332-ed7c-4ce0-acfb-da29993788ce None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:53.217554 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6617332-ed7c-4ce0-acfb-da29993788ce None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:53.217835 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6617332-ed7c-4ce0-acfb-da29993788ce None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:53.222128 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 411/811] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:15:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:15:53.222128 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6617332-ed7c-4ce0-acfb-da29993788ce None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:53.273269 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:53.274533 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1da5fc-b4cd-4b2a-b5de-e7ae090fde5e None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:53.308754 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7578b65-a6a7-4b3a-9488-802318da6560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:53.309426 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7578b65-a6a7-4b3a-9488-802318da6560 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:53.309710 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7578b65-a6a7-4b3a-9488-802318da6560 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:53.309985 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7578b65-a6a7-4b3a-9488-802318da6560 None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:53.310753 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7578b65-a6a7-4b3a-9488-802318da6560 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:53.311060 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7578b65-a6a7-4b3a-9488-802318da6560 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:53.312441 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7578b65-a6a7-4b3a-9488-802318da6560 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:53.317008 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 401/812] 10.222.0.66 () {56 vars in 1124 bytes} [Mon Jul 18 02:15:53 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:53.317783 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7578b65-a6a7-4b3a-9488-802318da6560 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:53.345778 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6617332-ed7c-4ce0-acfb-da29993788ce None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:53.380593 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d3b24e-2935-4d05-a4aa-12ecadf7bee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:53.381254 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d3b24e-2935-4d05-a4aa-12ecadf7bee3 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:53.381537 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d3b24e-2935-4d05-a4aa-12ecadf7bee3 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:53.381819 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d3b24e-2935-4d05-a4aa-12ecadf7bee3 None admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:53.382567 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d3b24e-2935-4d05-a4aa-12ecadf7bee3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:53.382873 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d3b24e-2935-4d05-a4aa-12ecadf7bee3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:53.384250 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d3b24e-2935-4d05-a4aa-12ecadf7bee3 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:53.392603 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:53.401316 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 412/813] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 18 02:15:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:53.401316 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88adb618-0ac6-4e52-ad43-4bba1f07320a None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:53.401316 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88adb618-0ac6-4e52-ad43-4bba1f07320a None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:53.401316 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88adb618-0ac6-4e52-ad43-4bba1f07320a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:53.415213 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-88adb618-0ac6-4e52-ad43-4bba1f07320a None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:15:53.416414 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-88adb618-0ac6-4e52-ad43-4bba1f07320a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:15:53.442506 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7578b65-a6a7-4b3a-9488-802318da6560 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:53.481071 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3d2dc7-3210-4b0f-bc28-31d61f2269ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:53.481856 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3d2dc7-3210-4b0f-bc28-31d61f2269ab None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:53.482382 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3d2dc7-3210-4b0f-bc28-31d61f2269ab None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:53.482518 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3d2dc7-3210-4b0f-bc28-31d61f2269ab None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:53.483216 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3d2dc7-3210-4b0f-bc28-31d61f2269ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:53.483525 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3d2dc7-3210-4b0f-bc28-31d61f2269ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:53.484948 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3d2dc7-3210-4b0f-bc28-31d61f2269ab None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:53.487563 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 413/814] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:15:53 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 02:15:53.488351 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88adb618-0ac6-4e52-ad43-4bba1f07320a None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:53.508240 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 402/815] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 18 02:15:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:53.509442 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3d2dc7-3210-4b0f-bc28-31d61f2269ab None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:53.520206 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:53.594959 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88adb618-0ac6-4e52-ad43-4bba1f07320a None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:53.617191 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3d2dc7-3210-4b0f-bc28-31d61f2269ab None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:53.637438 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b222c238-3497-4ea1-b28e-50f7fb50c0c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:53.638254 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b222c238-3497-4ea1-b28e-50f7fb50c0c4 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:53.638545 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b222c238-3497-4ea1-b28e-50f7fb50c0c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:53.638826 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b222c238-3497-4ea1-b28e-50f7fb50c0c4 None admin] PATH_INFO: `/v3/services` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:53.639086 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:53.639531 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b222c238-3497-4ea1-b28e-50f7fb50c0c4 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:53.639842 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b222c238-3497-4ea1-b28e-50f7fb50c0c4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:53.641293 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b222c238-3497-4ea1-b28e-50f7fb50c0c4 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:53.646460 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 414/816] 10.222.0.66 () {56 vars in 1124 bytes} [Mon Jul 18 02:15:53 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:53.647311 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b222c238-3497-4ea1-b28e-50f7fb50c0c4 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:53.659110 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b7b8a5-3ce2-462a-a846-56dfb1f0a6a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:53.660188 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b7b8a5-3ce2-462a-a846-56dfb1f0a6a6 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:53.660188 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b7b8a5-3ce2-462a-a846-56dfb1f0a6a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:53.660454 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b7b8a5-3ce2-462a-a846-56dfb1f0a6a6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:53.667964 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b7b8a5-3ce2-462a-a846-56dfb1f0a6a6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:53.669423 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b7b8a5-3ce2-462a-a846-56dfb1f0a6a6 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:53.684572 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-42b7b8a5-3ce2-462a-a846-56dfb1f0a6a6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 02:15:53.684572 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:53.684572 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:53.684572 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:53.684572 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:53.684572 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:53.684572 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:53.684572 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:53.684572 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:53.684572 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:53.684572 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:53.684572 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:53.684572 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 02:15:53.684572 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 02:15:53.684572 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 02:15:53.685492 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 02:15:53.685492 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:53.685492 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:53.685492 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:53.685492 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:53.685492 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:53.685492 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:53.685492 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:53.685492 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:53.685492 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:53.685492 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:53.685492 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:53.685492 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:53.685492 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:53.685492 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:53.685492 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:53.685492 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:53.686456 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 02:15:53.686456 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 02:15:53.686456 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 02:15:53.686456 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 02:15:53.686456 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 02:15:53.686456 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 02:15:53.686456 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 02:15:53.686456 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:15:53.686456 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 403/817] 10.222.0.66 () {56 vars in 1111 bytes} [Mon Jul 18 02:15:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:53.687279 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b7b8a5-3ce2-462a-a846-56dfb1f0a6a6 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:53.752395 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:53.761781 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b222c238-3497-4ea1-b28e-50f7fb50c0c4 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:53.791366 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b7b8a5-3ce2-462a-a846-56dfb1f0a6a6 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:53.797506 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c519bc7a-c701-4969-a661-357c9731bb5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:53.798562 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c519bc7a-c701-4969-a661-357c9731bb5f None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:53.799043 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c519bc7a-c701-4969-a661-357c9731bb5f None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:53.799496 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c519bc7a-c701-4969-a661-357c9731bb5f None admin] PATH_INFO: `/v3/users/nova` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:53.806945 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c519bc7a-c701-4969-a661-357c9731bb5f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:53.809285 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c519bc7a-c701-4969-a661-357c9731bb5f None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:53.824644 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-c519bc7a-c701-4969-a661-357c9731bb5f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 18 02:15:53.824644 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:53.824644 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:53.824644 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:53.824644 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:53.824644 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:53.824644 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:53.824644 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:53.824644 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:53.824644 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:53.824644 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:53.824644 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:53.824644 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 02:15:53.824644 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 02:15:53.824644 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 02:15:53.824644 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 02:15:53.825770 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:53.825770 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:53.825770 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 02:15:53.825770 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 02:15:53.825770 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 02:15:53.825770 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 02:15:53.825770 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:53.825770 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:53.825770 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:53.825770 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:53.825770 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:53.825770 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:53.825770 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:53.825770 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:53.825770 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:53.825770 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:53.825770 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:53.825770 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:53.826931 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:53.826931 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:53.826931 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 02:15:53.826931 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 02:15:53.826931 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 02:15:53.826931 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 02:15:53.826931 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 02:15:53.826931 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 02:15:53.826931 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 18 02:15:53.826931 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:15:53.828799 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 415/818] 10.222.0.66 () {56 vars in 1093 bytes} [Mon Jul 18 02:15:53 2022] GET /identity/v3/users/nova => generated 82 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:53.829075 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c6440a-098c-4c28-a1e8-c05e63f54c29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:53.829873 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c6440a-098c-4c28-a1e8-c05e63f54c29 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:53.830060 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c519bc7a-c701-4969-a661-357c9731bb5f None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:53.830211 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c6440a-098c-4c28-a1e8-c05e63f54c29 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:53.830455 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c6440a-098c-4c28-a1e8-c05e63f54c29 None admin] PATH_INFO: `/v3/services/network` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:53.831106 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c6440a-098c-4c28-a1e8-c05e63f54c29 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:53.832475 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c6440a-098c-4c28-a1e8-c05e63f54c29 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:53.845193 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-06c6440a-098c-4c28-a1e8-c05e63f54c29 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 18 02:15:53.845193 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:53.845193 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 02:15:53.845193 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 02:15:53.845193 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 02:15:53.845193 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 02:15:53.845193 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 02:15:53.845193 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 02:15:53.845193 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 18 02:15:53.845193 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:15:53.845193 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 02:15:53.845193 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:15:53.845193 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:53.845193 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:53.845193 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:53.845193 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:53.845193 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:53.846540 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:53.846540 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:53.846540 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:53.846540 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:53.846540 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:53.846540 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:53.846540 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 02:15:53.846540 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 02:15:53.846540 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 02:15:53.846540 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 02:15:53.846540 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:53.846540 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:53.846540 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:53.846540 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:53.846540 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:53.846540 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:53.847536 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:53.847536 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:53.847536 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:53.847536 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:53.847536 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:53.847536 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:53.847536 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:53.847536 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:53.847536 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:53.847536 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:53.847536 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 02:15:53.847536 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 02:15:53.847536 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 18 02:15:53.847536 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:15:53.847536 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 404/819] 10.222.0.66 () {56 vars in 1111 bytes} [Mon Jul 18 02:15:53 2022] GET /identity/v3/services/network => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:53.848337 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c6440a-098c-4c28-a1e8-c05e63f54c29 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:53.902120 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:53.947544 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c519bc7a-c701-4969-a661-357c9731bb5f None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:53.959679 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c6440a-098c-4c28-a1e8-c05e63f54c29 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:53.983015 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ee9f53-a1ff-4525-b1a8-fd595d3930fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:53.983781 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ee9f53-a1ff-4525-b1a8-fd595d3930fc None admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:53.984123 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ee9f53-a1ff-4525-b1a8-fd595d3930fc None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:53.984402 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ee9f53-a1ff-4525-b1a8-fd595d3930fc None admin] PATH_INFO: `/v3/endpoints` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:53.985140 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ee9f53-a1ff-4525-b1a8-fd595d3930fc None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:53.986528 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ee9f53-a1ff-4525-b1a8-fd595d3930fc None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:53.999065 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507d8dab-0286-4b11-882e-8999d6c191a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:53.999728 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507d8dab-0286-4b11-882e-8999d6c191a5 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:54.000016 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507d8dab-0286-4b11-882e-8999d6c191a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:54.000295 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507d8dab-0286-4b11-882e-8999d6c191a5 None admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:54.001080 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507d8dab-0286-4b11-882e-8999d6c191a5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:54.001476 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507d8dab-0286-4b11-882e-8999d6c191a5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:54.002885 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507d8dab-0286-4b11-882e-8999d6c191a5 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:54.015815 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 405/820] 10.222.0.66 () {56 vars in 1148 bytes} [Mon Jul 18 02:15:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:54.016644 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507d8dab-0286-4b11-882e-8999d6c191a5 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:54.093045 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 416/821] 10.222.0.66 () {60 vars in 1144 bytes} [Mon Jul 18 02:15:53 2022] POST /identity/v3/endpoints => generated 346 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:15:54.094700 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ee9f53-a1ff-4525-b1a8-fd595d3930fc None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:54.117850 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:54.124013 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507d8dab-0286-4b11-882e-8999d6c191a5 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:54.127515 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:54.185469 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100b6711-1c3b-4ea8-8fda-d6bfdce86ff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:54.187188 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100b6711-1c3b-4ea8-8fda-d6bfdce86ff0 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:54.187677 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100b6711-1c3b-4ea8-8fda-d6bfdce86ff0 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:54.188218 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100b6711-1c3b-4ea8-8fda-d6bfdce86ff0 None admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:54.189263 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100b6711-1c3b-4ea8-8fda-d6bfdce86ff0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:54.189971 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100b6711-1c3b-4ea8-8fda-d6bfdce86ff0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:54.192629 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100b6711-1c3b-4ea8-8fda-d6bfdce86ff0 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:54.219289 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 406/822] 10.222.0.66 () {56 vars in 1133 bytes} [Mon Jul 18 02:15:54 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:54.220540 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100b6711-1c3b-4ea8-8fda-d6bfdce86ff0 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:54.236025 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ee9f53-a1ff-4525-b1a8-fd595d3930fc None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:54.282582 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0977fa2a-27aa-4f4b-a68d-0ec03d708c1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:54.283259 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0977fa2a-27aa-4f4b-a68d-0ec03d708c1e None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:54.283547 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0977fa2a-27aa-4f4b-a68d-0ec03d708c1e None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:54.283826 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0977fa2a-27aa-4f4b-a68d-0ec03d708c1e None admin] PATH_INFO: `/v3/services` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:54.284671 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0977fa2a-27aa-4f4b-a68d-0ec03d708c1e None admin] RBAC: Adding query filter params (name=network) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:54.284985 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0977fa2a-27aa-4f4b-a68d-0ec03d708c1e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:54.287296 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0977fa2a-27aa-4f4b-a68d-0ec03d708c1e None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:54.292737 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 417/823] 10.222.0.66 () {56 vars in 1112 bytes} [Mon Jul 18 02:15:54 2022] GET /identity/v3/services?name=network => generated 124 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:54.293671 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0977fa2a-27aa-4f4b-a68d-0ec03d708c1e None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:54.348597 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100b6711-1c3b-4ea8-8fda-d6bfdce86ff0 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:54.379792 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:54.388644 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe3b91e-f87d-4ce5-9633-ef24ecc6ab1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:54.389722 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe3b91e-f87d-4ce5-9633-ef24ecc6ab1c None admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:54.390102 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe3b91e-f87d-4ce5-9633-ef24ecc6ab1c None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:54.390462 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe3b91e-f87d-4ce5-9633-ef24ecc6ab1c None admin] PATH_INFO: `/v3/projects/76f7c0ae36ae49b09dcae4c3e9a875f4/users/b4300622cbd347dd8f5227bb960e98d0/roles/d4aa3324f0c84887b41d0dc4acf7f30e` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:54.392715 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe3b91e-f87d-4ce5-9633-ef24ecc6ab1c None admin] RBAC: Authorizing `identity:create_grant(project_id=76f7c0ae36ae49b09dcae4c3e9a875f4, user_id=b4300622cbd347dd8f5227bb960e98d0, role_id=d4aa3324f0c84887b41d0dc4acf7f30e)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:54.394556 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe3b91e-f87d-4ce5-9633-ef24ecc6ab1c None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:54.412648 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0977fa2a-27aa-4f4b-a68d-0ec03d708c1e None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:54.451207 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f22fb1d-c635-4448-9f60-fdc1008aea8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:54.451839 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f22fb1d-c635-4448-9f60-fdc1008aea8b None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:54.452164 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f22fb1d-c635-4448-9f60-fdc1008aea8b None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:54.452488 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f22fb1d-c635-4448-9f60-fdc1008aea8b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:54.512905 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 407/824] 10.222.0.66 () {58 vars in 1439 bytes} [Mon Jul 18 02:15:54 2022] PUT /identity/v3/projects/76f7c0ae36ae49b09dcae4c3e9a875f4/users/b4300622cbd347dd8f5227bb960e98d0/roles/d4aa3324f0c84887b41d0dc4acf7f30e => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:15:54.514647 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe3b91e-f87d-4ce5-9633-ef24ecc6ab1c None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:54.532641 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f22fb1d-c635-4448-9f60-fdc1008aea8b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:54.534669 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f22fb1d-c635-4448-9f60-fdc1008aea8b None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:54.554489 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:54.682704 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:54.717184 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe3b91e-f87d-4ce5-9633-ef24ecc6ab1c None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:54.718153 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 418/825] 10.222.0.66 () {58 vars in 1282 bytes} [Mon Jul 18 02:15:54 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 02:15:54.724479 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f22fb1d-c635-4448-9f60-fdc1008aea8b None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:54.758284 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac2dc9e-48e1-4559-927d-f8c2f9a45f0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:54.759356 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac2dc9e-48e1-4559-927d-f8c2f9a45f0d None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:54.759845 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac2dc9e-48e1-4559-927d-f8c2f9a45f0d None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:54.760321 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac2dc9e-48e1-4559-927d-f8c2f9a45f0d None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:54.761782 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac2dc9e-48e1-4559-927d-f8c2f9a45f0d None admin] RBAC: Adding query filter params (type=network) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:54.762319 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac2dc9e-48e1-4559-927d-f8c2f9a45f0d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:54.763940 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac2dc9e-48e1-4559-927d-f8c2f9a45f0d None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:54.769804 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 408/826] 10.222.0.66 () {56 vars in 1112 bytes} [Mon Jul 18 02:15:54 2022] GET /identity/v3/services?type=network => generated 350 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:54.775882 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac2dc9e-48e1-4559-927d-f8c2f9a45f0d None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:54.905223 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:54.925577 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac2dc9e-48e1-4559-927d-f8c2f9a45f0d None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:54.957781 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f22fb1d-c635-4448-9f60-fdc1008aea8b None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:54.963148 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fba56f-7468-4a8a-a6c3-821116a18a61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:54.964637 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fba56f-7468-4a8a-a6c3-821116a18a61 None admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:54.965151 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fba56f-7468-4a8a-a6c3-821116a18a61 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:54.965710 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fba56f-7468-4a8a-a6c3-821116a18a61 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:54.966741 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fba56f-7468-4a8a-a6c3-821116a18a61 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:54.968407 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fba56f-7468-4a8a-a6c3-821116a18a61 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:54.995657 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1a2e1b-0d91-42bc-b230-252b814a6d10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:54.996499 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1a2e1b-0d91-42bc-b230-252b814a6d10 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:54.996785 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1a2e1b-0d91-42bc-b230-252b814a6d10 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:54.997058 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1a2e1b-0d91-42bc-b230-252b814a6d10 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:55.008140 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1a2e1b-0d91-42bc-b230-252b814a6d10 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:55.009611 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1a2e1b-0d91-42bc-b230-252b814a6d10 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:55.027707 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-0c1a2e1b-0d91-42bc-b230-252b814a6d10 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:15:55.027707 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:55.027707 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:55.027707 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:55.027707 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:55.027707 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:55.027707 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:55.027707 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:55.027707 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:55.027707 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:55.027707 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:55.027707 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:55.027707 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 02:15:55.027707 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 02:15:55.027707 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 02:15:55.028644 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 02:15:55.028644 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:55.028644 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:55.028644 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:55.028644 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:55.028644 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:55.028644 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:55.028644 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:55.028644 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:55.028644 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:55.028644 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:55.028644 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:55.028644 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:55.028644 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:55.028644 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:55.028644 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:55.028644 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:55.029575 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 02:15:55.029575 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 02:15:55.029575 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:15:55.029575 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:15:55.033445 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 419/827] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 18 02:15:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 02:15:55.034930 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03280f0c-7607-40ce-8659-3b7f1d24a5cd None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:55.035214 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03280f0c-7607-40ce-8659-3b7f1d24a5cd None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:55.035524 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03280f0c-7607-40ce-8659-3b7f1d24a5cd None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:55.036976 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 420/828] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:15:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:15:55.038342 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03280f0c-7607-40ce-8659-3b7f1d24a5cd None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:55.046332 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 409/829] 10.222.0.66 () {60 vars in 1144 bytes} [Mon Jul 18 02:15:54 2022] POST /identity/v3/endpoints => generated 327 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 02:15:55.048476 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8b9bc7-0f0c-4645-8509-005dba81817c None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:55.049028 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8b9bc7-0f0c-4645-8509-005dba81817c None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:55.049548 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8b9bc7-0f0c-4645-8509-005dba81817c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:55.068818 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:55.083320 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-fb8b9bc7-0f0c-4645-8509-005dba81817c None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:15:55.084898 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-fb8b9bc7-0f0c-4645-8509-005dba81817c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:15:55.178396 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03280f0c-7607-40ce-8659-3b7f1d24a5cd None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:55.219605 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddf7527-76f5-46b6-bbe6-1dc4a4a711e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:55.220220 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddf7527-76f5-46b6-bbe6-1dc4a4a711e4 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:55.220499 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddf7527-76f5-46b6-bbe6-1dc4a4a711e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:55.220775 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddf7527-76f5-46b6-bbe6-1dc4a4a711e4 None admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:55.222007 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddf7527-76f5-46b6-bbe6-1dc4a4a711e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:55.222346 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddf7527-76f5-46b6-bbe6-1dc4a4a711e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:55.223891 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddf7527-76f5-46b6-bbe6-1dc4a4a711e4 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:55.246208 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 421/830] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 18 02:15:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:55.250386 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddf7527-76f5-46b6-bbe6-1dc4a4a711e4 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:55.252430 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 410/831] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:15:55 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 02:15:55.277526 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8b9bc7-0f0c-4645-8509-005dba81817c None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:55.356526 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:55.383166 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddf7527-76f5-46b6-bbe6-1dc4a4a711e4 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:55.391489 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:55.408349 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8b9bc7-0f0c-4645-8509-005dba81817c None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:55.427446 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb175ba4-6f2c-4a8c-a5b0-fd8b1ef80a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:55.428308 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb175ba4-6f2c-4a8c-a5b0-fd8b1ef80a94 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:55.428664 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb175ba4-6f2c-4a8c-a5b0-fd8b1ef80a94 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:55.428909 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb175ba4-6f2c-4a8c-a5b0-fd8b1ef80a94 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:55.446292 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb175ba4-6f2c-4a8c-a5b0-fd8b1ef80a94 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:55.449244 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb175ba4-6f2c-4a8c-a5b0-fd8b1ef80a94 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:55.466927 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d39fe4-4e28-4c92-a449-efbd7a476aa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:55.467793 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d39fe4-4e28-4c92-a449-efbd7a476aa5 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:55.468085 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d39fe4-4e28-4c92-a449-efbd7a476aa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:55.468363 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d39fe4-4e28-4c92-a449-efbd7a476aa5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:55.468860 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-eb175ba4-6f2c-4a8c-a5b0-fd8b1ef80a94 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 02:15:55.468860 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:55.468860 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:55.468860 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:55.468860 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:55.468860 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:55.468860 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:55.468860 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:55.468860 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:55.468860 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:55.468860 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:55.468860 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:55.468860 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 02:15:55.468860 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 02:15:55.468860 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 02:15:55.469999 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 02:15:55.469999 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:55.469999 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:55.469999 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:55.469999 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:55.469999 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:55.469999 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:55.469999 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:55.469999 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:55.469999 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:55.469999 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:55.469999 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:55.469999 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:55.469999 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:55.469999 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:55.469999 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:55.469999 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:55.471501 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 02:15:55.471501 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 02:15:55.471501 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 02:15:55.471501 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 02:15:55.471501 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 02:15:55.471501 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 02:15:55.471501 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 02:15:55.471501 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:15:55.473110 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 422/832] 10.222.0.66 () {56 vars in 1111 bytes} [Mon Jul 18 02:15:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 02:15:55.478799 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb175ba4-6f2c-4a8c-a5b0-fd8b1ef80a94 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:55.486214 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d39fe4-4e28-4c92-a449-efbd7a476aa5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 02:15:55.498497 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-e7d39fe4-4e28-4c92-a449-efbd7a476aa5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 02:15:55.498497 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:55.498497 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:55.498497 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:55.498497 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:55.498497 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:55.498497 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:55.498497 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:55.498497 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:55.498497 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:55.498497 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:55.498497 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:55.498497 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 02:15:55.498497 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 02:15:55.498497 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 02:15:55.498497 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 02:15:55.499688 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 02:15:55.499688 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 02:15:55.499688 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 02:15:55.499688 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 02:15:55.499688 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:55.499688 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:55.499688 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:55.499688 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:55.499688 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:55.499688 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:55.499688 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:55.499688 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:55.499688 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:55.499688 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:55.499688 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:55.499688 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:55.500746 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:55.500746 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:55.500746 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:55.500746 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:55.500746 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 02:15:55.500746 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 02:15:55.500746 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 02:15:55.500746 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 02:15:55.500746 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 02:15:55.500746 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 02:15:55.500746 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 02:15:55.500746 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:15:55.503551 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 411/833] 10.222.0.66 () {56 vars in 1102 bytes} [Mon Jul 18 02:15:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:55.509335 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d39fe4-4e28-4c92-a449-efbd7a476aa5 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:55.614394 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb175ba4-6f2c-4a8c-a5b0-fd8b1ef80a94 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:55.627464 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d39fe4-4e28-4c92-a449-efbd7a476aa5 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:55.652207 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a16d13-d92b-42ba-b628-cc5d21c33bc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:55.652859 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a16d13-d92b-42ba-b628-cc5d21c33bc8 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:55.653136 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a16d13-d92b-42ba-b628-cc5d21c33bc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:55.653401 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a16d13-d92b-42ba-b628-cc5d21c33bc8 None admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:55.654216 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a16d13-d92b-42ba-b628-cc5d21c33bc8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:55.654526 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a16d13-d92b-42ba-b628-cc5d21c33bc8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:55.656050 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a16d13-d92b-42ba-b628-cc5d21c33bc8 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:55.662733 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:55.667470 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:55.672618 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 423/834] 10.222.0.66 () {56 vars in 1148 bytes} [Mon Jul 18 02:15:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:55.676524 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f0ace2-4eff-4eef-aebb-0d3f882de4ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:55.676524 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f0ace2-4eff-4eef-aebb-0d3f882de4ea None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:55.676524 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f0ace2-4eff-4eef-aebb-0d3f882de4ea None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:55.676524 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f0ace2-4eff-4eef-aebb-0d3f882de4ea None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:55.676880 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f0ace2-4eff-4eef-aebb-0d3f882de4ea None admin] RBAC: Adding query filter params (name=service) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:55.676880 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f0ace2-4eff-4eef-aebb-0d3f882de4ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:55.677048 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f0ace2-4eff-4eef-aebb-0d3f882de4ea None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:55.678430 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a16d13-d92b-42ba-b628-cc5d21c33bc8 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:55.694133 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 412/835] 10.222.0.66 () {56 vars in 1103 bytes} [Mon Jul 18 02:15:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 02:15:55.697380 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f0ace2-4eff-4eef-aebb-0d3f882de4ea None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:55.789380 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a16d13-d92b-42ba-b628-cc5d21c33bc8 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:55.828511 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4b556a-4ab1-4f2f-a68c-4b594b436c5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:55.829160 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4b556a-4ab1-4f2f-a68c-4b594b436c5f None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:55.829437 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4b556a-4ab1-4f2f-a68c-4b594b436c5f None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:55.829707 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4b556a-4ab1-4f2f-a68c-4b594b436c5f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:55.830801 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4b556a-4ab1-4f2f-a68c-4b594b436c5f None admin] RBAC: Adding query filter params (role.id=d4aa3324f0c84887b41d0dc4acf7f30e, scope.project.id=76f7c0ae36ae49b09dcae4c3e9a875f4, user.id=80b138831bc54c7b986d50092d62d4af) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:55.830988 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4b556a-4ab1-4f2f-a68c-4b594b436c5f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:55.832775 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4b556a-4ab1-4f2f-a68c-4b594b436c5f None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:55.835135 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f0ace2-4eff-4eef-aebb-0d3f882de4ea None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:55.846462 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 424/836] 10.222.0.66 () {56 vars in 1374 bytes} [Mon Jul 18 02:15:55 2022] GET /identity/v3/role_assignments?user.id=80b138831bc54c7b986d50092d62d4af&scope.project.id=76f7c0ae36ae49b09dcae4c3e9a875f4&role.id=d4aa3324f0c84887b41d0dc4acf7f30e => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 02:15:55.877148 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e73233-ee4e-47b4-bd0f-78ae8441ba8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:55.878383 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e73233-ee4e-47b4-bd0f-78ae8441ba8a None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:55.878832 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e73233-ee4e-47b4-bd0f-78ae8441ba8a None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:55.879265 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e73233-ee4e-47b4-bd0f-78ae8441ba8a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:55.891731 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e73233-ee4e-47b4-bd0f-78ae8441ba8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:55.893536 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e73233-ee4e-47b4-bd0f-78ae8441ba8a None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:55.915177 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-74e73233-ee4e-47b4-bd0f-78ae8441ba8a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:15:55.915177 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:55.915177 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:55.915177 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:55.915177 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:55.915177 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:55.915177 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:55.915177 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:55.915177 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:55.915177 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:55.915177 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:55.915177 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:55.915177 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 02:15:55.915177 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 02:15:55.915177 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 02:15:55.916352 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 02:15:55.916352 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:55.916352 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:55.916352 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:55.916352 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:55.916352 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:55.916352 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:55.916352 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:55.916352 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:55.916352 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:55.916352 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:55.916352 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:55.916352 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:55.916352 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:55.916352 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:55.916352 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:55.916352 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:55.917530 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 02:15:55.917530 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 02:15:55.917530 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:15:55.917530 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:15:55.919602 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 413/837] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 18 02:15:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 02:15:55.930213 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4b556a-4ab1-4f2f-a68c-4b594b436c5f None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:56.016622 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:56.048519 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4b556a-4ab1-4f2f-a68c-4b594b436c5f None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:56.084529 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f990298-79b7-4870-9bc5-e45846ab3b39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:56.085201 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f990298-79b7-4870-9bc5-e45846ab3b39 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:56.085483 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f990298-79b7-4870-9bc5-e45846ab3b39 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:56.085760 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f990298-79b7-4870-9bc5-e45846ab3b39 None admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:56.086508 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f990298-79b7-4870-9bc5-e45846ab3b39 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:56.086816 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f990298-79b7-4870-9bc5-e45846ab3b39 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:56.088187 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f990298-79b7-4870-9bc5-e45846ab3b39 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:56.100857 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 425/838] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 18 02:15:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:56.106405 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e73233-ee4e-47b4-bd0f-78ae8441ba8a None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:56.151989 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:56.210991 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e73233-ee4e-47b4-bd0f-78ae8441ba8a None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:56.247706 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e1db97-dee4-4913-ba1d-05683874e8ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:56.248800 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e1db97-dee4-4913-ba1d-05683874e8ce None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:56.249248 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e1db97-dee4-4913-ba1d-05683874e8ce None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:56.249671 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e1db97-dee4-4913-ba1d-05683874e8ce None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:56.257377 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e1db97-dee4-4913-ba1d-05683874e8ce None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:56.259036 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e1db97-dee4-4913-ba1d-05683874e8ce None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:56.274890 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-c1e1db97-dee4-4913-ba1d-05683874e8ce None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 02:15:56.274890 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:56.274890 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:56.274890 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:56.274890 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:56.274890 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:56.274890 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:56.274890 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:56.274890 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:56.274890 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:56.274890 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:56.274890 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:56.274890 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 02:15:56.274890 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 02:15:56.274890 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 02:15:56.274890 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 02:15:56.276154 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:56.276154 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:56.276154 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 02:15:56.276154 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 02:15:56.276154 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 02:15:56.276154 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 02:15:56.276154 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:56.276154 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:56.276154 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:56.276154 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:56.276154 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:56.276154 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:56.276154 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:56.276154 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:56.276154 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:56.276154 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:56.276154 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:56.276154 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:56.277417 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:56.277417 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:56.277417 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 02:15:56.277417 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 02:15:56.277417 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 02:15:56.277417 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 02:15:56.277417 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 02:15:56.277417 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 02:15:56.277417 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 02:15:56.277417 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:15:56.279745 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 414/839] 10.222.0.66 () {56 vars in 1117 bytes} [Mon Jul 18 02:15:56 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:56.285295 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f990298-79b7-4870-9bc5-e45846ab3b39 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:56.384712 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cee56d0-6930-48af-8589-43ffb45a561c None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:56.384892 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cee56d0-6930-48af-8589-43ffb45a561c None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:56.385173 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cee56d0-6930-48af-8589-43ffb45a561c None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:56.388247 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 415/840] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:15:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 02:15:56.393619 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036e8805-7cfd-40ea-808c-7d804c8bb00f None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:56.393909 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036e8805-7cfd-40ea-808c-7d804c8bb00f None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:56.394220 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036e8805-7cfd-40ea-808c-7d804c8bb00f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:56.397299 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f990298-79b7-4870-9bc5-e45846ab3b39 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:56.415181 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-036e8805-7cfd-40ea-808c-7d804c8bb00f None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:15:56.416889 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-036e8805-7cfd-40ea-808c-7d804c8bb00f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:15:56.423493 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:56.435470 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6befdd93-f8aa-4034-9794-2d1762569d2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:56.436236 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6befdd93-f8aa-4034-9794-2d1762569d2a None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:56.436519 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6befdd93-f8aa-4034-9794-2d1762569d2a None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:56.436790 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6befdd93-f8aa-4034-9794-2d1762569d2a None admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:56.437630 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6befdd93-f8aa-4034-9794-2d1762569d2a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:56.437934 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6befdd93-f8aa-4034-9794-2d1762569d2a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:56.439607 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6befdd93-f8aa-4034-9794-2d1762569d2a None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:56.460352 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 426/841] 10.222.0.66 () {56 vars in 1149 bytes} [Mon Jul 18 02:15:56 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:56.466232 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6befdd93-f8aa-4034-9794-2d1762569d2a None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:56.502557 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 416/842] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:15:56 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 02:15:56.529291 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036e8805-7cfd-40ea-808c-7d804c8bb00f None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:56.567547 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:56.590294 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6befdd93-f8aa-4034-9794-2d1762569d2a None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:56.635967 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888684eb-a174-4d65-876f-e1a17113dbd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:56.636902 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888684eb-a174-4d65-876f-e1a17113dbd2 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:56.637374 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888684eb-a174-4d65-876f-e1a17113dbd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:56.637786 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888684eb-a174-4d65-876f-e1a17113dbd2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:56.659270 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036e8805-7cfd-40ea-808c-7d804c8bb00f None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:56.694803 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:56.704625 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2071ca-2a88-4c8d-a93c-601a01b3c9d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:56.705776 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2071ca-2a88-4c8d-a93c-601a01b3c9d7 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:56.706071 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2071ca-2a88-4c8d-a93c-601a01b3c9d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:56.709179 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2071ca-2a88-4c8d-a93c-601a01b3c9d7 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:56.709179 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2071ca-2a88-4c8d-a93c-601a01b3c9d7 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:56.709360 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2071ca-2a88-4c8d-a93c-601a01b3c9d7 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:56.715739 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888684eb-a174-4d65-876f-e1a17113dbd2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:56.717343 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888684eb-a174-4d65-876f-e1a17113dbd2 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:56.724340 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-2c2071ca-2a88-4c8d-a93c-601a01b3c9d7 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 18 02:15:56.724340 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:56.724340 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 02:15:56.724340 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 02:15:56.724340 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 02:15:56.724340 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 02:15:56.724340 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 02:15:56.724340 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 02:15:56.724340 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 18 02:15:56.724340 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:15:56.724340 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 02:15:56.724340 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:15:56.724340 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:56.724340 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:56.724340 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:56.724340 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:56.724340 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:56.725518 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:56.725518 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:56.725518 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:56.725518 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:56.725518 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:56.725518 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:56.725518 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 02:15:56.725518 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 02:15:56.725518 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 02:15:56.725518 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 02:15:56.725518 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:56.725518 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:56.725518 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:56.725518 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:56.725518 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:56.725518 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:56.726622 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:56.726622 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:56.726622 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:56.726622 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:56.726622 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:56.726622 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:56.726622 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:56.726622 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:56.726622 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:56.726622 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:56.726622 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 02:15:56.726622 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 02:15:56.726622 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 18 02:15:56.726622 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:15:56.730236 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 417/843] 10.222.0.66 () {56 vars in 1114 bytes} [Mon Jul 18 02:15:56 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 02:15:56.732237 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2071ca-2a88-4c8d-a93c-601a01b3c9d7 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:56.830065 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 427/844] 10.222.0.66 () {58 vars in 1282 bytes} [Mon Jul 18 02:15:56 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 18 02:15:56.832081 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693aa699-4477-49c0-a70b-0fb5aa4cd848 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:56.832547 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693aa699-4477-49c0-a70b-0fb5aa4cd848 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:56.832996 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693aa699-4477-49c0-a70b-0fb5aa4cd848 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:56.834637 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 428/845] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:15:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 02:15:56.835527 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693aa699-4477-49c0-a70b-0fb5aa4cd848 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:56.845093 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:56.851203 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2071ca-2a88-4c8d-a93c-601a01b3c9d7 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:56.889998 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edff38a5-445e-4560-b69f-d231005f9b82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:56.890641 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edff38a5-445e-4560-b69f-d231005f9b82 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:56.891043 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edff38a5-445e-4560-b69f-d231005f9b82 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:56.891324 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edff38a5-445e-4560-b69f-d231005f9b82 None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:56.892149 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edff38a5-445e-4560-b69f-d231005f9b82 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:56.892463 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edff38a5-445e-4560-b69f-d231005f9b82 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:56.894018 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edff38a5-445e-4560-b69f-d231005f9b82 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:56.898884 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 418/846] 10.222.0.66 () {56 vars in 1114 bytes} [Mon Jul 18 02:15:56 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:56.900037 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c6645e-b28f-41cb-9451-e60f1c54ae73 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:56.900322 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c6645e-b28f-41cb-9451-e60f1c54ae73 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:56.900598 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c6645e-b28f-41cb-9451-e60f1c54ae73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:56.918875 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-d8c6645e-b28f-41cb-9451-e60f1c54ae73 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:15:56.920298 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-d8c6645e-b28f-41cb-9451-e60f1c54ae73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:15:56.931499 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:56.951627 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693aa699-4477-49c0-a70b-0fb5aa4cd848 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:56.988693 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea14d99e-c736-4348-828b-b2a2037d4495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:56.989636 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea14d99e-c736-4348-828b-b2a2037d4495 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:56.990046 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea14d99e-c736-4348-828b-b2a2037d4495 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:56.990417 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea14d99e-c736-4348-828b-b2a2037d4495 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:56.996148 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 419/847] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:15:56 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 02:15:56.997266 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c6645e-b28f-41cb-9451-e60f1c54ae73 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:57.001906 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea14d99e-c736-4348-828b-b2a2037d4495 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:57.003528 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea14d99e-c736-4348-828b-b2a2037d4495 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:57.023104 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-ea14d99e-c736-4348-828b-b2a2037d4495 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:15:57.023104 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:57.023104 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:57.023104 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:57.023104 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:57.023104 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:57.023104 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:57.023104 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:57.023104 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:57.023104 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:57.023104 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:57.023104 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:57.023104 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 02:15:57.023104 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 02:15:57.023104 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 02:15:57.024655 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 02:15:57.024655 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:57.024655 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:57.024655 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:57.024655 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:57.024655 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:57.024655 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:57.024655 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:57.024655 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:57.024655 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:57.024655 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:57.024655 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:57.024655 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:57.024655 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:57.024655 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:57.024655 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:57.024655 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:57.025759 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 02:15:57.025759 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 02:15:57.025759 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:15:57.025759 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:15:57.025759 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 429/848] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 18 02:15:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:57.025759 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea14d99e-c736-4348-828b-b2a2037d4495 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:57.106614 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:57.116297 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c6645e-b28f-41cb-9451-e60f1c54ae73 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:57.136553 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea14d99e-c736-4348-828b-b2a2037d4495 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:57.152551 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb242396-ddd2-43d8-9b78-97f278d48408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:57.153229 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb242396-ddd2-43d8-9b78-97f278d48408 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:57.153517 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb242396-ddd2-43d8-9b78-97f278d48408 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:57.153794 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb242396-ddd2-43d8-9b78-97f278d48408 None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:57.154541 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb242396-ddd2-43d8-9b78-97f278d48408 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:57.154848 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb242396-ddd2-43d8-9b78-97f278d48408 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:57.156232 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb242396-ddd2-43d8-9b78-97f278d48408 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:57.161359 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 420/849] 10.222.0.66 () {56 vars in 1114 bytes} [Mon Jul 18 02:15:56 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:57.161949 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb242396-ddd2-43d8-9b78-97f278d48408 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:57.190534 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3fbea8-46c9-4845-a850-86a33adf9ec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:57.192585 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3fbea8-46c9-4845-a850-86a33adf9ec8 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:57.193140 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3fbea8-46c9-4845-a850-86a33adf9ec8 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:57.193923 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3fbea8-46c9-4845-a850-86a33adf9ec8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:57.197483 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:57.212224 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3fbea8-46c9-4845-a850-86a33adf9ec8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 02:15:57.222911 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-1f3fbea8-46c9-4845-a850-86a33adf9ec8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 02:15:57.222911 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:57.222911 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:57.222911 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:57.222911 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:57.222911 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:57.222911 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:57.222911 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:57.222911 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:57.222911 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:57.222911 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:57.222911 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:57.222911 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 02:15:57.222911 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 02:15:57.222911 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 02:15:57.222911 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 02:15:57.223952 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 02:15:57.223952 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 02:15:57.223952 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 02:15:57.223952 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 02:15:57.223952 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:57.223952 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:57.223952 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:57.223952 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:57.223952 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:57.223952 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:57.223952 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:57.223952 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:57.223952 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:57.223952 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:57.223952 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:57.223952 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:57.225086 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:57.225086 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:57.225086 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:57.225086 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:57.225086 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 02:15:57.225086 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 02:15:57.225086 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 02:15:57.225086 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 02:15:57.225086 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 02:15:57.225086 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 02:15:57.225086 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 02:15:57.225086 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:15:57.225086 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 430/850] 10.222.0.66 () {56 vars in 1096 bytes} [Mon Jul 18 02:15:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:57.231185 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3fbea8-46c9-4845-a850-86a33adf9ec8 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:57.300240 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb242396-ddd2-43d8-9b78-97f278d48408 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:57.344464 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e2d3a4-100a-4b84-84ba-2fc3be101eff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:57.345232 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e2d3a4-100a-4b84-84ba-2fc3be101eff None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:57.345620 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e2d3a4-100a-4b84-84ba-2fc3be101eff None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:57.345966 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e2d3a4-100a-4b84-84ba-2fc3be101eff None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:57.346856 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e2d3a4-100a-4b84-84ba-2fc3be101eff None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:57.347192 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e2d3a4-100a-4b84-84ba-2fc3be101eff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:57.364546 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e2d3a4-100a-4b84-84ba-2fc3be101eff None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:57.364709 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3fbea8-46c9-4845-a850-86a33adf9ec8 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:57.364811 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 421/851] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 18 02:15:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:57.370889 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e2d3a4-100a-4b84-84ba-2fc3be101eff None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:57.392218 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:57.393658 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869a7c0a-9c3e-4b2a-80ec-43f84e6cd545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:57.399657 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869a7c0a-9c3e-4b2a-80ec-43f84e6cd545 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:57.399657 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869a7c0a-9c3e-4b2a-80ec-43f84e6cd545 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:57.399657 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869a7c0a-9c3e-4b2a-80ec-43f84e6cd545 None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:57.399657 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869a7c0a-9c3e-4b2a-80ec-43f84e6cd545 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:57.399657 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869a7c0a-9c3e-4b2a-80ec-43f84e6cd545 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:57.399657 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869a7c0a-9c3e-4b2a-80ec-43f84e6cd545 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:57.411147 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 431/852] 10.222.0.66 () {56 vars in 1099 bytes} [Mon Jul 18 02:15:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:57.416238 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869a7c0a-9c3e-4b2a-80ec-43f84e6cd545 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:57.492963 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:57.506981 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e2d3a4-100a-4b84-84ba-2fc3be101eff None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:57.552156 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869a7c0a-9c3e-4b2a-80ec-43f84e6cd545 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:57.554336 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69305759-a09b-4e8f-beab-4cf3cf834393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:57.556151 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69305759-a09b-4e8f-beab-4cf3cf834393 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:57.556611 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69305759-a09b-4e8f-beab-4cf3cf834393 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:57.556975 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69305759-a09b-4e8f-beab-4cf3cf834393 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:57.566433 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69305759-a09b-4e8f-beab-4cf3cf834393 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:57.568746 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69305759-a09b-4e8f-beab-4cf3cf834393 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:57.589345 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-69305759-a09b-4e8f-beab-4cf3cf834393 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 02:15:57.589345 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:57.589345 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:57.589345 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:57.589345 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:57.589345 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:57.589345 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:57.589345 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:57.589345 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:57.589345 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:57.589345 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:57.589345 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:57.589345 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 02:15:57.589345 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 02:15:57.589345 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 02:15:57.590333 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 02:15:57.590333 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:57.590333 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:57.590333 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:57.590333 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:57.590333 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:57.590333 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:57.590333 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:57.590333 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:57.590333 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:57.590333 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:57.590333 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:57.590333 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:57.590333 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:57.590333 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:57.590333 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:57.590333 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:57.591266 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 02:15:57.591266 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 02:15:57.591266 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 02:15:57.591266 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 02:15:57.591266 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 02:15:57.591266 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 02:15:57.591266 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 02:15:57.591266 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:15:57.594227 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 422/853] 10.222.0.66 () {56 vars in 1111 bytes} [Mon Jul 18 02:15:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:57.596352 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69305759-a09b-4e8f-beab-4cf3cf834393 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:57.611030 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e073c786-4adb-4f20-b7bc-6fde0a18354e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:57.611740 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e073c786-4adb-4f20-b7bc-6fde0a18354e None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:57.612025 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e073c786-4adb-4f20-b7bc-6fde0a18354e None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:57.612299 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e073c786-4adb-4f20-b7bc-6fde0a18354e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:57.626662 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e073c786-4adb-4f20-b7bc-6fde0a18354e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:57.628600 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e073c786-4adb-4f20-b7bc-6fde0a18354e None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:57.648792 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-e073c786-4adb-4f20-b7bc-6fde0a18354e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:15:57.648792 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:57.648792 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:57.648792 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:57.648792 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:57.648792 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:57.648792 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:57.648792 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:57.648792 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:57.648792 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:57.648792 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:57.648792 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:57.648792 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 02:15:57.648792 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 02:15:57.648792 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 02:15:57.649770 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 02:15:57.649770 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:57.649770 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:57.649770 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:57.649770 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:57.649770 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:57.649770 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:57.649770 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:57.649770 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:57.649770 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:57.649770 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:57.649770 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:57.649770 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:57.649770 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:57.649770 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:57.649770 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:57.649770 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:57.651158 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 02:15:57.651158 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 02:15:57.651158 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:15:57.651158 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:15:57.653333 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 432/854] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 18 02:15:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:57.659577 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e073c786-4adb-4f20-b7bc-6fde0a18354e None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:57.691011 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:57.717480 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69305759-a09b-4e8f-beab-4cf3cf834393 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:57.763671 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8186ea6e-52ee-4087-b7e1-4c05c5b9b753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:57.764695 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8186ea6e-52ee-4087-b7e1-4c05c5b9b753 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:57.765135 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8186ea6e-52ee-4087-b7e1-4c05c5b9b753 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:57.765816 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8186ea6e-52ee-4087-b7e1-4c05c5b9b753 None admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:57.767332 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8186ea6e-52ee-4087-b7e1-4c05c5b9b753 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:57.768076 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8186ea6e-52ee-4087-b7e1-4c05c5b9b753 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:57.770427 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8186ea6e-52ee-4087-b7e1-4c05c5b9b753 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:57.792751 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 423/855] 10.222.0.66 () {56 vars in 1148 bytes} [Mon Jul 18 02:15:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 02:15:57.792934 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e073c786-4adb-4f20-b7bc-6fde0a18354e None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:57.794245 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8186ea6e-52ee-4087-b7e1-4c05c5b9b753 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:57.804295 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:57.841824 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeb4c80-c6d3-4ab4-b3a7-200984cf6e14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:57.843171 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeb4c80-c6d3-4ab4-b3a7-200984cf6e14 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:57.843689 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeb4c80-c6d3-4ab4-b3a7-200984cf6e14 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:57.844436 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeb4c80-c6d3-4ab4-b3a7-200984cf6e14 None admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:57.845878 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbeb4c80-c6d3-4ab4-b3a7-200984cf6e14 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:57.846970 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbeb4c80-c6d3-4ab4-b3a7-200984cf6e14 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:57.849661 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbeb4c80-c6d3-4ab4-b3a7-200984cf6e14 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:57.868675 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 433/856] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 18 02:15:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:57.875124 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeb4c80-c6d3-4ab4-b3a7-200984cf6e14 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:57.913406 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8186ea6e-52ee-4087-b7e1-4c05c5b9b753 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:57.963886 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384ccc61-8805-4c58-828c-e90cbf538c71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:57.965158 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384ccc61-8805-4c58-828c-e90cbf538c71 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:57.965651 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384ccc61-8805-4c58-828c-e90cbf538c71 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:57.966477 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384ccc61-8805-4c58-828c-e90cbf538c71 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:57.967499 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384ccc61-8805-4c58-828c-e90cbf538c71 None admin] RBAC: Adding query filter params (role.id=eea94daf3312421cb12aa8abfa73160e, scope.project.id=76f7c0ae36ae49b09dcae4c3e9a875f4, user.id=768f872096584d4093fc616d547aabb6) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:57.968014 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384ccc61-8805-4c58-828c-e90cbf538c71 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:57.970199 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384ccc61-8805-4c58-828c-e90cbf538c71 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:57.973228 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:57.986113 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeb4c80-c6d3-4ab4-b3a7-200984cf6e14 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:57.986613 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 424/857] 10.222.0.66 () {56 vars in 1374 bytes} [Mon Jul 18 02:15:57 2022] GET /identity/v3/role_assignments?user.id=768f872096584d4093fc616d547aabb6&scope.project.id=76f7c0ae36ae49b09dcae4c3e9a875f4&role.id=eea94daf3312421cb12aa8abfa73160e => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 02:15:58.018674 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9c2982-16c1-496f-a01a-929eb83cb50b None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:58.018963 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9c2982-16c1-496f-a01a-929eb83cb50b None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:58.019340 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9c2982-16c1-496f-a01a-929eb83cb50b None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:58.021003 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 425/858] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:15:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:15:58.027715 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2533af1b-ed18-4d2d-adf9-fc9030c5ac05 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:58.028023 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2533af1b-ed18-4d2d-adf9-fc9030c5ac05 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:58.028263 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2533af1b-ed18-4d2d-adf9-fc9030c5ac05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:58.028593 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e096a1b0-2efc-4d8c-905b-f6a45c181511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:58.030028 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e096a1b0-2efc-4d8c-905b-f6a45c181511 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:58.030584 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e096a1b0-2efc-4d8c-905b-f6a45c181511 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:58.030869 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e096a1b0-2efc-4d8c-905b-f6a45c181511 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:58.039051 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e096a1b0-2efc-4d8c-905b-f6a45c181511 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:58.040525 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e096a1b0-2efc-4d8c-905b-f6a45c181511 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:58.047152 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-2533af1b-ed18-4d2d-adf9-fc9030c5ac05 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:15:58.048758 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-2533af1b-ed18-4d2d-adf9-fc9030c5ac05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:15:58.056640 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-e096a1b0-2efc-4d8c-905b-f6a45c181511 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 18 02:15:58.056640 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:58.056640 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:58.056640 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:58.056640 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:58.056640 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:58.056640 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:58.056640 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:58.056640 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:58.056640 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:58.056640 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:58.056640 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:58.056640 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 02:15:58.056640 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 02:15:58.056640 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 02:15:58.056640 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 02:15:58.058317 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:58.058317 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:58.058317 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 02:15:58.058317 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 02:15:58.058317 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 02:15:58.058317 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 02:15:58.058317 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:58.058317 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:58.058317 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:58.058317 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:58.058317 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:58.058317 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:58.058317 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:58.058317 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:58.058317 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:58.058317 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:58.058317 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:58.058317 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:58.059294 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:58.059294 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:58.059294 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 02:15:58.059294 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 02:15:58.059294 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 02:15:58.059294 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 02:15:58.059294 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 02:15:58.059294 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 02:15:58.059294 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 18 02:15:58.059294 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:15:58.063892 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 434/859] 10.222.0.66 () {56 vars in 1096 bytes} [Mon Jul 18 02:15:57 2022] GET /identity/v3/users/swift => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:58.065278 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e096a1b0-2efc-4d8c-905b-f6a45c181511 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:58.098580 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:58.140291 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 426/860] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:15:58 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 02:15:58.163171 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2533af1b-ed18-4d2d-adf9-fc9030c5ac05 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:58.190188 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e096a1b0-2efc-4d8c-905b-f6a45c181511 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:58.227970 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9974330-8ff3-4a33-a9a7-c804fcbd3a0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:58.229704 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9974330-8ff3-4a33-a9a7-c804fcbd3a0b None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:58.230251 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9974330-8ff3-4a33-a9a7-c804fcbd3a0b None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:58.230800 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9974330-8ff3-4a33-a9a7-c804fcbd3a0b None admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:58.231859 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9974330-8ff3-4a33-a9a7-c804fcbd3a0b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:58.232377 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9974330-8ff3-4a33-a9a7-c804fcbd3a0b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:58.234050 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9974330-8ff3-4a33-a9a7-c804fcbd3a0b None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:58.258186 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 435/861] 10.222.0.66 () {56 vars in 1135 bytes} [Mon Jul 18 02:15:58 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 02:15:58.261423 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9974330-8ff3-4a33-a9a7-c804fcbd3a0b None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:58.299604 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2533af1b-ed18-4d2d-adf9-fc9030c5ac05 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:58.336865 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:58.348438 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3464e77c-2938-4a8f-b213-16cd0cd7b4f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:58.349295 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3464e77c-2938-4a8f-b213-16cd0cd7b4f9 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:58.349585 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3464e77c-2938-4a8f-b213-16cd0cd7b4f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:58.360677 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3464e77c-2938-4a8f-b213-16cd0cd7b4f9 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:58.360677 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3464e77c-2938-4a8f-b213-16cd0cd7b4f9 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:58.360677 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3464e77c-2938-4a8f-b213-16cd0cd7b4f9 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:58.364770 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:58.370205 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-3464e77c-2938-4a8f-b213-16cd0cd7b4f9 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 18 02:15:58.370205 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:58.370205 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 02:15:58.370205 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 02:15:58.370205 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 02:15:58.370205 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 02:15:58.370205 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 02:15:58.370205 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 02:15:58.370205 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 18 02:15:58.370205 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:15:58.370205 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 02:15:58.370205 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:15:58.370205 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:58.370205 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:58.370205 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:58.370205 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:58.371258 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:58.371258 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:58.371258 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:58.371258 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:58.371258 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:58.371258 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:58.371258 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:58.371258 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 02:15:58.371258 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 02:15:58.371258 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 02:15:58.371258 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 02:15:58.371258 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:58.371258 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:58.371258 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:58.371258 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:58.371258 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:58.372262 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:58.372262 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:58.372262 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:58.372262 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:58.372262 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:58.372262 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:58.372262 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:58.372262 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:58.372262 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:58.372262 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:58.372262 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:58.372262 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 02:15:58.372262 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 02:15:58.372262 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 18 02:15:58.372262 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:15:58.372262 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 427/862] 10.222.0.66 () {56 vars in 1132 bytes} [Mon Jul 18 02:15:58 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 02:15:58.377378 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3464e77c-2938-4a8f-b213-16cd0cd7b4f9 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:58.398320 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9974330-8ff3-4a33-a9a7-c804fcbd3a0b None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:58.442967 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e8ab2b-4482-4ccd-aa5a-5c9f63e611b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:58.443675 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e8ab2b-4482-4ccd-aa5a-5c9f63e611b6 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:58.443962 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e8ab2b-4482-4ccd-aa5a-5c9f63e611b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:58.444285 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e8ab2b-4482-4ccd-aa5a-5c9f63e611b6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:58.507340 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3464e77c-2938-4a8f-b213-16cd0cd7b4f9 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:58.529470 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e8ab2b-4482-4ccd-aa5a-5c9f63e611b6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:58.531510 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e8ab2b-4482-4ccd-aa5a-5c9f63e611b6 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:58.542661 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995a2943-ffa0-4b07-95c3-314416007f49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:58.543365 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995a2943-ffa0-4b07-95c3-314416007f49 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:58.543654 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995a2943-ffa0-4b07-95c3-314416007f49 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:58.543934 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995a2943-ffa0-4b07-95c3-314416007f49 None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:58.544645 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995a2943-ffa0-4b07-95c3-314416007f49 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:58.544954 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995a2943-ffa0-4b07-95c3-314416007f49 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:58.546609 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995a2943-ffa0-4b07-95c3-314416007f49 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:58.551686 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 428/863] 10.222.0.66 () {56 vars in 1126 bytes} [Mon Jul 18 02:15:58 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 02:15:58.557334 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995a2943-ffa0-4b07-95c3-314416007f49 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:58.633935 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:58.638996 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:58.663718 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 436/864] 10.222.0.66 () {58 vars in 1282 bytes} [Mon Jul 18 02:15:58 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 02:15:58.667306 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e8ab2b-4482-4ccd-aa5a-5c9f63e611b6 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:58.695429 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995a2943-ffa0-4b07-95c3-314416007f49 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:58.733172 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa565ed-b24d-4592-b807-6ab4f00803d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:58.734275 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa565ed-b24d-4592-b807-6ab4f00803d3 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:58.734758 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa565ed-b24d-4592-b807-6ab4f00803d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:58.735278 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa565ed-b24d-4592-b807-6ab4f00803d3 None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:58.736290 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa565ed-b24d-4592-b807-6ab4f00803d3 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:58.736791 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa565ed-b24d-4592-b807-6ab4f00803d3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:58.738407 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa565ed-b24d-4592-b807-6ab4f00803d3 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:58.749477 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 429/865] 10.222.0.66 () {56 vars in 1126 bytes} [Mon Jul 18 02:15:58 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:58.779358 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e8ab2b-4482-4ccd-aa5a-5c9f63e611b6 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:58.818201 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8e9e21-f8e4-41d2-9446-5018aa294b10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:58.819015 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8e9e21-f8e4-41d2-9446-5018aa294b10 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:58.819300 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8e9e21-f8e4-41d2-9446-5018aa294b10 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:58.819575 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8e9e21-f8e4-41d2-9446-5018aa294b10 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:58.831545 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8e9e21-f8e4-41d2-9446-5018aa294b10 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:58.833032 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8e9e21-f8e4-41d2-9446-5018aa294b10 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:58.853082 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-7e8e9e21-f8e4-41d2-9446-5018aa294b10 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:15:58.853082 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:58.853082 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:58.853082 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:58.853082 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:58.853082 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:58.853082 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:58.853082 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:58.853082 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:58.853082 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:58.853082 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:58.853082 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:58.853082 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 02:15:58.853082 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 02:15:58.853082 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 02:15:58.854265 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 02:15:58.854265 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:58.854265 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:58.854265 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:58.854265 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:58.854265 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:58.854265 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:58.854265 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:58.854265 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:58.854265 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:58.854265 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:58.854265 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:58.854265 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:58.854265 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:58.854265 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:58.854265 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:58.854265 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:58.855469 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 02:15:58.855469 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 02:15:58.855469 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:15:58.855469 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:15:58.855469 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 437/866] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 18 02:15:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:58.861695 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa565ed-b24d-4592-b807-6ab4f00803d3 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:58.974686 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa565ed-b24d-4592-b807-6ab4f00803d3 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:58.998341 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:59.012859 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d2c67f-a11e-4d01-8ad4-18e7f1d35725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:59.013528 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d2c67f-a11e-4d01-8ad4-18e7f1d35725 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:59.013807 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d2c67f-a11e-4d01-8ad4-18e7f1d35725 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:59.014099 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d2c67f-a11e-4d01-8ad4-18e7f1d35725 None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:59.014832 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d2c67f-a11e-4d01-8ad4-18e7f1d35725 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:59.015142 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d2c67f-a11e-4d01-8ad4-18e7f1d35725 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:59.016499 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d2c67f-a11e-4d01-8ad4-18e7f1d35725 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:59.034369 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 430/867] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 18 02:15:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:59.035624 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8e9e21-f8e4-41d2-9446-5018aa294b10 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:59.077786 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:59.140288 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8e9e21-f8e4-41d2-9446-5018aa294b10 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:59.176395 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc7ccce-cfff-4fbb-a8f2-9a41803c8049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:59.177552 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc7ccce-cfff-4fbb-a8f2-9a41803c8049 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:59.178039 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc7ccce-cfff-4fbb-a8f2-9a41803c8049 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:59.178557 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc7ccce-cfff-4fbb-a8f2-9a41803c8049 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:59.186034 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc7ccce-cfff-4fbb-a8f2-9a41803c8049 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:59.187813 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc7ccce-cfff-4fbb-a8f2-9a41803c8049 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:59.202639 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-4dc7ccce-cfff-4fbb-a8f2-9a41803c8049 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 02:15:59.202639 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:15:59.202639 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:15:59.202639 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:15:59.202639 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:15:59.202639 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:15:59.202639 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:15:59.202639 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:15:59.202639 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:15:59.202639 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:15:59.202639 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:15:59.202639 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:15:59.202639 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 02:15:59.202639 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 02:15:59.202639 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 02:15:59.203566 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 02:15:59.203566 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:15:59.203566 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:15:59.203566 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:15:59.203566 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:15:59.203566 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:15:59.203566 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:15:59.203566 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:15:59.203566 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:15:59.203566 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:15:59.203566 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:15:59.203566 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:15:59.203566 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:15:59.203566 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:15:59.203566 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:15:59.203566 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:15:59.203566 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:15:59.204468 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 02:15:59.204468 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 02:15:59.204468 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 02:15:59.204468 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 02:15:59.204468 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 02:15:59.204468 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 02:15:59.204468 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 02:15:59.204468 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:15:59.206946 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 438/868] 10.222.0.66 () {56 vars in 1111 bytes} [Mon Jul 18 02:15:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:15:59.214113 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d2c67f-a11e-4d01-8ad4-18e7f1d35725 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:59.242059 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76da43e-d655-473f-8fe1-825689b7b82a None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:59.242388 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76da43e-d655-473f-8fe1-825689b7b82a None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:59.242667 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76da43e-d655-473f-8fe1-825689b7b82a None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:59.244044 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 439/869] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:15:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 02:15:59.251375 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a429f9c0-6775-481f-9d66-090bababdd8f None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:59.251662 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a429f9c0-6775-481f-9d66-090bababdd8f None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:59.251937 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a429f9c0-6775-481f-9d66-090bababdd8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:59.274608 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-a429f9c0-6775-481f-9d66-090bababdd8f None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:15:59.275881 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-a429f9c0-6775-481f-9d66-090bababdd8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:15:59.341728 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d2c67f-a11e-4d01-8ad4-18e7f1d35725 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:59.350339 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 440/870] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:15:59 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 02:15:59.370002 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a429f9c0-6775-481f-9d66-090bababdd8f None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:59.381428 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68520def-2e32-48fb-a0e6-d1e536a24934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:59.382636 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68520def-2e32-48fb-a0e6-d1e536a24934 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:59.383041 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68520def-2e32-48fb-a0e6-d1e536a24934 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:59.383363 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68520def-2e32-48fb-a0e6-d1e536a24934 None admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:59.384205 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68520def-2e32-48fb-a0e6-d1e536a24934 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:59.384683 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68520def-2e32-48fb-a0e6-d1e536a24934 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:59.387028 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68520def-2e32-48fb-a0e6-d1e536a24934 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:59.402376 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 431/871] 10.222.0.66 () {56 vars in 1148 bytes} [Mon Jul 18 02:15:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:59.407813 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68520def-2e32-48fb-a0e6-d1e536a24934 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:15:59.412117 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:59.467773 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:15:59.501933 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a429f9c0-6775-481f-9d66-090bababdd8f None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:59.529643 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68520def-2e32-48fb-a0e6-d1e536a24934 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:15:59.552198 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15cd2df-beec-4981-9715-749ac9869b68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:59.552959 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15cd2df-beec-4981-9715-749ac9869b68 None admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:59.553310 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15cd2df-beec-4981-9715-749ac9869b68 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:59.553649 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15cd2df-beec-4981-9715-749ac9869b68 None admin] PATH_INFO: `/v3/services` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:59.554750 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15cd2df-beec-4981-9715-749ac9869b68 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:59.557358 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15cd2df-beec-4981-9715-749ac9869b68 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:59.568349 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d26def-f7f8-4f26-8926-6523ba6d7721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:15:59.572665 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d26def-f7f8-4f26-8926-6523ba6d7721 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:15:59.572665 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d26def-f7f8-4f26-8926-6523ba6d7721 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:15:59.572665 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d26def-f7f8-4f26-8926-6523ba6d7721 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:15:59.572665 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d26def-f7f8-4f26-8926-6523ba6d7721 None admin] RBAC: Adding query filter params (role.id=d4aa3324f0c84887b41d0dc4acf7f30e, scope.project.id=76f7c0ae36ae49b09dcae4c3e9a875f4, user.id=b4300622cbd347dd8f5227bb960e98d0) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:15:59.572665 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d26def-f7f8-4f26-8926-6523ba6d7721 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:15:59.572665 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d26def-f7f8-4f26-8926-6523ba6d7721 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:15:59.579051 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 432/872] 10.222.0.66 () {56 vars in 1374 bytes} [Mon Jul 18 02:15:59 2022] GET /identity/v3/role_assignments?user.id=b4300622cbd347dd8f5227bb960e98d0&scope.project.id=76f7c0ae36ae49b09dcae4c3e9a875f4&role.id=d4aa3324f0c84887b41d0dc4acf7f30e => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:15:59.642219 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 441/873] 10.222.0.66 () {60 vars in 1141 bytes} [Mon Jul 18 02:15:59 2022] POST /identity/v3/services => generated 251 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:16:00.230303 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0483e01-ba2c-443c-865f-503f33ab15c1 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:00.231476 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0483e01-ba2c-443c-865f-503f33ab15c1 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:00.231970 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0483e01-ba2c-443c-865f-503f33ab15c1 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:00.233824 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 433/874] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:16:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:16:00.239477 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b19f8be-14f6-4599-b177-1a7c18ad95eb None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:00.240295 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b19f8be-14f6-4599-b177-1a7c18ad95eb None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:00.240799 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b19f8be-14f6-4599-b177-1a7c18ad95eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:00.258845 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-5b19f8be-14f6-4599-b177-1a7c18ad95eb None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:16:00.260577 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-5b19f8be-14f6-4599-b177-1a7c18ad95eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:16:00.338750 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 442/875] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:16:00 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 02:16:00.361347 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0483e01-ba2c-443c-865f-503f33ab15c1 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:00.473032 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0483e01-ba2c-443c-865f-503f33ab15c1 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:00.475402 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:00.508456 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c8c18b-56bc-416a-911c-da7888733441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:00.509192 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c8c18b-56bc-416a-911c-da7888733441 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:00.509476 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c8c18b-56bc-416a-911c-da7888733441 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:00.509805 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c8c18b-56bc-416a-911c-da7888733441 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:00.521107 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c8c18b-56bc-416a-911c-da7888733441 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 02:16:00.530168 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-b2c8c18b-56bc-416a-911c-da7888733441 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 02:16:00.530168 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:16:00.530168 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:16:00.530168 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:16:00.530168 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:16:00.530168 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:16:00.530168 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:16:00.530168 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:16:00.530168 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:16:00.530168 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:16:00.530168 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:16:00.530168 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:16:00.530168 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 02:16:00.530168 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 02:16:00.530168 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 02:16:00.530168 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 02:16:00.531154 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 02:16:00.531154 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 02:16:00.531154 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 02:16:00.531154 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 02:16:00.531154 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:16:00.531154 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:16:00.531154 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:16:00.531154 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:16:00.531154 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:16:00.531154 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:16:00.531154 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:16:00.531154 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:16:00.531154 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:16:00.531154 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:16:00.531154 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:16:00.531154 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:16:00.532048 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:16:00.532048 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:16:00.532048 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:16:00.532048 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:16:00.532048 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 02:16:00.532048 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 02:16:00.532048 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 02:16:00.532048 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 02:16:00.532048 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 02:16:00.532048 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 02:16:00.532048 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 02:16:00.532048 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:16:00.532048 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 434/876] 10.222.0.66 () {56 vars in 1120 bytes} [Mon Jul 18 02:16:00 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:16:00.542781 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b19f8be-14f6-4599-b177-1a7c18ad95eb None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:00.559468 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:00.648580 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b19f8be-14f6-4599-b177-1a7c18ad95eb None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:00.684600 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e484b2e-c3d1-4d64-a109-3f5b6d341b2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:00.685271 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e484b2e-c3d1-4d64-a109-3f5b6d341b2d None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:00.685549 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e484b2e-c3d1-4d64-a109-3f5b6d341b2d None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:00.685821 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e484b2e-c3d1-4d64-a109-3f5b6d341b2d None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:00.686598 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e484b2e-c3d1-4d64-a109-3f5b6d341b2d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:16:00.686903 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e484b2e-c3d1-4d64-a109-3f5b6d341b2d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:00.688288 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e484b2e-c3d1-4d64-a109-3f5b6d341b2d None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:00.703588 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 443/877] 10.222.0.66 () {56 vars in 1115 bytes} [Mon Jul 18 02:16:00 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:16:00.708947 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c8c18b-56bc-416a-911c-da7888733441 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:00.818789 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c8c18b-56bc-416a-911c-da7888733441 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:00.855691 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9b6672-db9e-43f3-9226-bc509bde71d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:00.856498 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9b6672-db9e-43f3-9226-bc509bde71d5 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:00.856840 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9b6672-db9e-43f3-9226-bc509bde71d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:00.857121 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9b6672-db9e-43f3-9226-bc509bde71d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:00.872135 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9b6672-db9e-43f3-9226-bc509bde71d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:00.873883 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9b6672-db9e-43f3-9226-bc509bde71d5 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:00.892287 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-5b9b6672-db9e-43f3-9226-bc509bde71d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:16:00.892287 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:16:00.892287 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:16:00.892287 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:16:00.892287 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:16:00.892287 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:16:00.892287 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:16:00.892287 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:16:00.892287 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:16:00.892287 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:16:00.892287 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:16:00.892287 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:16:00.892287 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 02:16:00.892287 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 02:16:00.892287 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 02:16:00.893240 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 02:16:00.893240 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:16:00.893240 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:16:00.893240 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:16:00.893240 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:16:00.893240 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:16:00.893240 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:16:00.893240 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:16:00.893240 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:16:00.893240 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:16:00.893240 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:16:00.893240 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:16:00.893240 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:16:00.893240 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:16:00.893240 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:16:00.893240 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:16:00.893240 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:16:00.894199 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 02:16:00.894199 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 02:16:00.894199 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:16:00.894199 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:16:00.896456 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 435/878] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 18 02:16:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:16:00.901598 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855067fd-39e7-4bc0-9078-cf9cecd6d06a None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:00.901598 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855067fd-39e7-4bc0-9078-cf9cecd6d06a None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:00.901598 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855067fd-39e7-4bc0-9078-cf9cecd6d06a None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:00.901898 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 444/879] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:16:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:16:00.902938 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9b6672-db9e-43f3-9226-bc509bde71d5 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:00.907381 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07625a72-4a30-4e5f-8d02-689ccc36cbf4 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:00.907895 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07625a72-4a30-4e5f-8d02-689ccc36cbf4 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:00.908389 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07625a72-4a30-4e5f-8d02-689ccc36cbf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:00.924236 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:00.930595 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-07625a72-4a30-4e5f-8d02-689ccc36cbf4 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:16:00.932909 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-07625a72-4a30-4e5f-8d02-689ccc36cbf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:16:01.009443 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9b6672-db9e-43f3-9226-bc509bde71d5 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:01.018958 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 445/880] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:16:00 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 02:16:01.041463 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07625a72-4a30-4e5f-8d02-689ccc36cbf4 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:01.050309 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:01.051313 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaa7f1a-00e7-4a54-840f-f1960c21c9d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:01.052031 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaa7f1a-00e7-4a54-840f-f1960c21c9d9 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:01.052317 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaa7f1a-00e7-4a54-840f-f1960c21c9d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:01.052597 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaa7f1a-00e7-4a54-840f-f1960c21c9d9 None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:01.053321 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eaa7f1a-00e7-4a54-840f-f1960c21c9d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:16:01.053628 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eaa7f1a-00e7-4a54-840f-f1960c21c9d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:01.055027 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eaa7f1a-00e7-4a54-840f-f1960c21c9d9 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:01.067949 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 436/881] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 18 02:16:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:16:01.073765 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaa7f1a-00e7-4a54-840f-f1960c21c9d9 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:01.159373 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:01.168570 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07625a72-4a30-4e5f-8d02-689ccc36cbf4 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:01.183661 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaa7f1a-00e7-4a54-840f-f1960c21c9d9 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:01.214517 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07a9ba0-11c6-4a3b-9e03-9b31cc869d19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:01.215854 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07a9ba0-11c6-4a3b-9e03-9b31cc869d19 None admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:01.216353 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07a9ba0-11c6-4a3b-9e03-9b31cc869d19 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:01.216732 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07a9ba0-11c6-4a3b-9e03-9b31cc869d19 None admin] PATH_INFO: `/v3/services` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:01.217561 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07a9ba0-11c6-4a3b-9e03-9b31cc869d19 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:01.219097 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07a9ba0-11c6-4a3b-9e03-9b31cc869d19 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:01.220019 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a66cea-aeb3-4978-8c93-5d2fe64c2e50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:01.220837 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a66cea-aeb3-4978-8c93-5d2fe64c2e50 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:01.221119 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a66cea-aeb3-4978-8c93-5d2fe64c2e50 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:01.221395 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a66cea-aeb3-4978-8c93-5d2fe64c2e50 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:01.230168 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a66cea-aeb3-4978-8c93-5d2fe64c2e50 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:01.235178 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a66cea-aeb3-4978-8c93-5d2fe64c2e50 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:01.247340 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-26a66cea-aeb3-4978-8c93-5d2fe64c2e50 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 02:16:01.247340 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:16:01.247340 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:16:01.247340 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:16:01.247340 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:16:01.247340 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:16:01.247340 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:16:01.247340 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:16:01.247340 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:16:01.247340 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:16:01.247340 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:16:01.247340 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:16:01.247340 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 02:16:01.247340 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 02:16:01.247340 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 02:16:01.247340 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 02:16:01.248391 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:16:01.248391 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:16:01.248391 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 02:16:01.248391 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 02:16:01.248391 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 02:16:01.248391 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 02:16:01.248391 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:16:01.248391 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:16:01.248391 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:16:01.248391 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:16:01.248391 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:16:01.248391 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:16:01.248391 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:16:01.248391 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:16:01.248391 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:16:01.248391 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:16:01.248391 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:16:01.248391 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:16:01.249375 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:16:01.249375 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:16:01.249375 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 02:16:01.249375 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 02:16:01.249375 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 02:16:01.249375 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 02:16:01.249375 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 02:16:01.249375 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 02:16:01.249375 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 02:16:01.249375 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:16:01.250623 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 437/882] 10.222.0.66 () {56 vars in 1117 bytes} [Mon Jul 18 02:16:01 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 02:16:01.254063 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a66cea-aeb3-4978-8c93-5d2fe64c2e50 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:01.338810 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 446/883] 10.222.0.66 () {60 vars in 1141 bytes} [Mon Jul 18 02:16:01 2022] POST /identity/v3/services => generated 269 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 02:16:01.377945 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a66cea-aeb3-4978-8c93-5d2fe64c2e50 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:01.420993 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:01.427069 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e751c34-10f6-4dc0-9ce4-08593dcb901d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:01.428097 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e751c34-10f6-4dc0-9ce4-08593dcb901d None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:01.428750 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e751c34-10f6-4dc0-9ce4-08593dcb901d None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:01.429246 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e751c34-10f6-4dc0-9ce4-08593dcb901d None admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:01.430385 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e751c34-10f6-4dc0-9ce4-08593dcb901d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:16:01.430875 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e751c34-10f6-4dc0-9ce4-08593dcb901d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:01.433020 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e751c34-10f6-4dc0-9ce4-08593dcb901d None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:01.458447 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 438/884] 10.222.0.66 () {56 vars in 1149 bytes} [Mon Jul 18 02:16:01 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 02:16:01.460621 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07a9ba0-11c6-4a3b-9e03-9b31cc869d19 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:01.538972 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:01.581327 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07a9ba0-11c6-4a3b-9e03-9b31cc869d19 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:01.626363 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e743b70a-da53-4c35-8c2f-712ba1091a3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:01.627036 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e743b70a-da53-4c35-8c2f-712ba1091a3c None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:01.627725 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e743b70a-da53-4c35-8c2f-712ba1091a3c None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:01.628335 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e743b70a-da53-4c35-8c2f-712ba1091a3c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:01.702536 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e743b70a-da53-4c35-8c2f-712ba1091a3c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:01.704003 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e743b70a-da53-4c35-8c2f-712ba1091a3c None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:01.719534 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd606b4-94d3-4292-8dd1-67b207942e96 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:01.719832 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd606b4-94d3-4292-8dd1-67b207942e96 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:01.720099 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd606b4-94d3-4292-8dd1-67b207942e96 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:01.722726 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 439/885] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:16:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 02:16:01.726789 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2da7c16-6a63-4e56-bf27-076afe894a7e None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:01.727072 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2da7c16-6a63-4e56-bf27-076afe894a7e None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:01.727348 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2da7c16-6a63-4e56-bf27-076afe894a7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:01.745566 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-a2da7c16-6a63-4e56-bf27-076afe894a7e None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:16:01.746781 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-a2da7c16-6a63-4e56-bf27-076afe894a7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:16:01.787396 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:01.825325 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 440/886] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:16:01 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 02:16:01.826822 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6319efb6-3cfc-414d-ab18-8cc31ad4f908 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:01.827114 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6319efb6-3cfc-414d-ab18-8cc31ad4f908 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:01.827389 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6319efb6-3cfc-414d-ab18-8cc31ad4f908 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:01.829131 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 441/887] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:16:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:16:01.833817 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d97f837-ec75-4cf7-bbf4-5ae5205c20ed None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:01.834259 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d97f837-ec75-4cf7-bbf4-5ae5205c20ed None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:01.834677 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d97f837-ec75-4cf7-bbf4-5ae5205c20ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:01.836417 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 447/888] 10.222.0.66 () {58 vars in 1282 bytes} [Mon Jul 18 02:16:01 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 02:16:01.842923 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e743b70a-da53-4c35-8c2f-712ba1091a3c None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:01.858822 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-3d97f837-ec75-4cf7-bbf4-5ae5205c20ed None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:16:01.860092 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-3d97f837-ec75-4cf7-bbf4-5ae5205c20ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:16:01.930130 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 442/889] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:16:01 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (3 switches on core 0) Jul 18 02:16:01.930130 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d97f837-ec75-4cf7-bbf4-5ae5205c20ed None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:01.955569 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e743b70a-da53-4c35-8c2f-712ba1091a3c None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:01.983842 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:01.990669 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14df7774-b240-4d6e-aed1-ff11a9b0de72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:01.991455 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14df7774-b240-4d6e-aed1-ff11a9b0de72 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:01.991736 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14df7774-b240-4d6e-aed1-ff11a9b0de72 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:01.992006 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14df7774-b240-4d6e-aed1-ff11a9b0de72 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:02.002737 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14df7774-b240-4d6e-aed1-ff11a9b0de72 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:02.004187 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14df7774-b240-4d6e-aed1-ff11a9b0de72 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:02.023191 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-14df7774-b240-4d6e-aed1-ff11a9b0de72 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:16:02.023191 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:16:02.023191 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:16:02.023191 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:16:02.023191 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:16:02.023191 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:16:02.023191 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:16:02.023191 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:16:02.023191 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:16:02.023191 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:16:02.023191 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:16:02.023191 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:16:02.023191 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 02:16:02.023191 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 02:16:02.023191 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 02:16:02.024141 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 02:16:02.024141 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:16:02.024141 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:16:02.024141 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:16:02.024141 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:16:02.024141 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:16:02.024141 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:16:02.024141 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:16:02.024141 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:16:02.024141 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:16:02.024141 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:16:02.024141 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:16:02.024141 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:16:02.024141 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:16:02.024141 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:16:02.024141 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:16:02.024141 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:16:02.025057 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 02:16:02.025057 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 02:16:02.025057 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:16:02.025057 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:16:02.025057 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 448/890] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 18 02:16:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:16:02.025380 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14df7774-b240-4d6e-aed1-ff11a9b0de72 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:02.046362 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:02.046561 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d97f837-ec75-4cf7-bbf4-5ae5205c20ed None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:02.082013 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5e77eb-7041-4ddf-8d47-1777dd7a6db6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:02.082843 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5e77eb-7041-4ddf-8d47-1777dd7a6db6 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:02.083132 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5e77eb-7041-4ddf-8d47-1777dd7a6db6 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:02.083392 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5e77eb-7041-4ddf-8d47-1777dd7a6db6 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:02.084060 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5e77eb-7041-4ddf-8d47-1777dd7a6db6 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:02.085436 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5e77eb-7041-4ddf-8d47-1777dd7a6db6 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:02.097458 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-bb5e77eb-7041-4ddf-8d47-1777dd7a6db6 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 02:16:02.097458 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:16:02.097458 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 02:16:02.097458 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 02:16:02.097458 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 02:16:02.097458 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 02:16:02.097458 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 02:16:02.097458 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 02:16:02.097458 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 02:16:02.097458 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:16:02.097458 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 02:16:02.097458 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:16:02.097458 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:16:02.097458 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:16:02.097458 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:16:02.097458 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:16:02.097458 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:16:02.098676 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:16:02.098676 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:16:02.098676 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:16:02.098676 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:16:02.098676 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:16:02.098676 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:16:02.098676 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 02:16:02.098676 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 02:16:02.098676 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 02:16:02.098676 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 02:16:02.098676 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:16:02.098676 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:16:02.098676 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:16:02.098676 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:16:02.098676 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:16:02.098676 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:16:02.099539 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:16:02.099539 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:16:02.099539 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:16:02.099539 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:16:02.099539 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:16:02.099539 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:16:02.099539 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:16:02.099539 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:16:02.099539 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:16:02.099539 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:16:02.099539 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 02:16:02.099539 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 02:16:02.099539 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 02:16:02.099539 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:16:02.099539 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 443/891] 10.222.0.66 () {56 vars in 1126 bytes} [Mon Jul 18 02:16:01 2022] GET /identity/v3/services/object-store => generated 93 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:16:02.100428 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5e77eb-7041-4ddf-8d47-1777dd7a6db6 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:02.136266 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14df7774-b240-4d6e-aed1-ff11a9b0de72 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:02.176133 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa20893-9cd7-4cda-9554-31675202e9b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:02.176929 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa20893-9cd7-4cda-9554-31675202e9b1 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:02.177209 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa20893-9cd7-4cda-9554-31675202e9b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:02.177480 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa20893-9cd7-4cda-9554-31675202e9b1 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:02.178165 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa20893-9cd7-4cda-9554-31675202e9b1 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:02.179538 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa20893-9cd7-4cda-9554-31675202e9b1 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:02.192648 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-1fa20893-9cd7-4cda-9554-31675202e9b1 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 18 02:16:02.192648 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:16:02.192648 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 02:16:02.192648 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 02:16:02.192648 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 02:16:02.192648 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 02:16:02.192648 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 02:16:02.192648 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 02:16:02.192648 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 18 02:16:02.192648 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:16:02.192648 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 02:16:02.192648 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:16:02.192648 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:16:02.192648 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:16:02.192648 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:16:02.192648 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:16:02.192648 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:16:02.193664 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:16:02.193664 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:16:02.193664 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:16:02.193664 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:16:02.193664 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:16:02.193664 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:16:02.193664 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 02:16:02.193664 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 02:16:02.193664 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 02:16:02.193664 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 02:16:02.193664 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:16:02.193664 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:16:02.193664 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:16:02.193664 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:16:02.193664 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:16:02.193664 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:16:02.194582 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:16:02.194582 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:16:02.194582 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:16:02.194582 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:16:02.194582 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:16:02.194582 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:16:02.194582 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:16:02.194582 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:16:02.194582 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:16:02.194582 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:16:02.194582 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 02:16:02.194582 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 02:16:02.194582 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 18 02:16:02.194582 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:16:02.194582 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 449/892] 10.222.0.66 () {56 vars in 1114 bytes} [Mon Jul 18 02:16:02 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:16:02.195999 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa20893-9cd7-4cda-9554-31675202e9b1 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:02.210720 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5e77eb-7041-4ddf-8d47-1777dd7a6db6 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:02.225920 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:02.247811 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f09359e-a41a-4c25-9693-1f019f1b6b2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:02.248470 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f09359e-a41a-4c25-9693-1f019f1b6b2f None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:02.248760 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f09359e-a41a-4c25-9693-1f019f1b6b2f None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:02.249038 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f09359e-a41a-4c25-9693-1f019f1b6b2f None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:02.249767 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f09359e-a41a-4c25-9693-1f019f1b6b2f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:16:02.250106 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f09359e-a41a-4c25-9693-1f019f1b6b2f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:02.251515 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f09359e-a41a-4c25-9693-1f019f1b6b2f None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:02.265287 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 444/893] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 18 02:16:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:16:02.266004 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f09359e-a41a-4c25-9693-1f019f1b6b2f None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:02.287654 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:02.308723 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa20893-9cd7-4cda-9554-31675202e9b1 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:02.345333 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7946f7-8cff-49a9-84f0-a13abb6e9d04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:02.346050 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7946f7-8cff-49a9-84f0-a13abb6e9d04 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:02.346491 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7946f7-8cff-49a9-84f0-a13abb6e9d04 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:02.346757 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7946f7-8cff-49a9-84f0-a13abb6e9d04 None admin] PATH_INFO: `/v3/services` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:02.347462 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7946f7-8cff-49a9-84f0-a13abb6e9d04 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:16:02.347891 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7946f7-8cff-49a9-84f0-a13abb6e9d04 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:02.349272 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7946f7-8cff-49a9-84f0-a13abb6e9d04 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:02.355227 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 450/894] 10.222.0.66 () {56 vars in 1122 bytes} [Mon Jul 18 02:16:02 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:16:02.356075 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7946f7-8cff-49a9-84f0-a13abb6e9d04 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:02.377870 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f09359e-a41a-4c25-9693-1f019f1b6b2f None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:02.412371 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f6f7c8-2790-4d75-ac15-10442c4e932e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:02.413187 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f6f7c8-2790-4d75-ac15-10442c4e932e None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:02.413469 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f6f7c8-2790-4d75-ac15-10442c4e932e None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:02.413737 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f6f7c8-2790-4d75-ac15-10442c4e932e None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:02.414458 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f6f7c8-2790-4d75-ac15-10442c4e932e None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:16:02.414768 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f6f7c8-2790-4d75-ac15-10442c4e932e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:02.416747 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f6f7c8-2790-4d75-ac15-10442c4e932e None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:02.426348 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 445/895] 10.222.0.66 () {56 vars in 1114 bytes} [Mon Jul 18 02:16:02 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:16:02.426348 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f6f7c8-2790-4d75-ac15-10442c4e932e None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:02.466160 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:02.469253 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7946f7-8cff-49a9-84f0-a13abb6e9d04 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:02.504340 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd6dc87-8bb0-4de5-82be-19c3781ac83d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:02.505134 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd6dc87-8bb0-4de5-82be-19c3781ac83d None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:02.505414 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd6dc87-8bb0-4de5-82be-19c3781ac83d None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:02.505690 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd6dc87-8bb0-4de5-82be-19c3781ac83d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:02.513240 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd6dc87-8bb0-4de5-82be-19c3781ac83d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:02.514725 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd6dc87-8bb0-4de5-82be-19c3781ac83d None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:02.528515 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f6f7c8-2790-4d75-ac15-10442c4e932e None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:02.529266 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-4bd6dc87-8bb0-4de5-82be-19c3781ac83d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 02:16:02.529266 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:16:02.529266 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:16:02.529266 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:16:02.529266 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:16:02.529266 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:16:02.529266 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:16:02.529266 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:16:02.529266 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:16:02.529266 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:16:02.529266 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:16:02.529266 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:16:02.529266 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 02:16:02.529266 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 02:16:02.529266 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 02:16:02.530248 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 02:16:02.530248 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:16:02.530248 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:16:02.530248 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:16:02.530248 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:16:02.530248 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:16:02.530248 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:16:02.530248 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:16:02.530248 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:16:02.530248 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:16:02.530248 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:16:02.530248 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:16:02.530248 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:16:02.530248 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:16:02.530248 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:16:02.530248 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:16:02.530248 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:16:02.531867 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 02:16:02.531867 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 02:16:02.531867 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 02:16:02.531867 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 02:16:02.531867 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 02:16:02.531867 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 02:16:02.531867 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 02:16:02.531867 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:16:02.531867 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 451/896] 10.222.0.66 () {56 vars in 1111 bytes} [Mon Jul 18 02:16:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:16:02.531867 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd6dc87-8bb0-4de5-82be-19c3781ac83d None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:02.544218 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:02.571659 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa49cb4-933b-4483-ad0b-f9e18fcd35ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:02.573026 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa49cb4-933b-4483-ad0b-f9e18fcd35ef None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:02.573315 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa49cb4-933b-4483-ad0b-f9e18fcd35ef None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:02.573592 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa49cb4-933b-4483-ad0b-f9e18fcd35ef None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:02.574340 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa49cb4-933b-4483-ad0b-f9e18fcd35ef None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:16:02.574649 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa49cb4-933b-4483-ad0b-f9e18fcd35ef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:02.576035 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa49cb4-933b-4483-ad0b-f9e18fcd35ef None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:02.582140 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 446/897] 10.222.0.66 () {56 vars in 1122 bytes} [Mon Jul 18 02:16:02 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:16:02.582140 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa49cb4-933b-4483-ad0b-f9e18fcd35ef None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:02.681345 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd6dc87-8bb0-4de5-82be-19c3781ac83d None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:02.702210 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa49cb4-933b-4483-ad0b-f9e18fcd35ef None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:02.704165 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:02.731168 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f8e40f-f7f2-4e41-b11b-2e552bbc12e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:02.731538 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f8e40f-f7f2-4e41-b11b-2e552bbc12e4 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:02.731927 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f8e40f-f7f2-4e41-b11b-2e552bbc12e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:02.732307 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f8e40f-f7f2-4e41-b11b-2e552bbc12e4 None admin] PATH_INFO: `/v3/services` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:02.733141 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f8e40f-f7f2-4e41-b11b-2e552bbc12e4 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:16:02.733544 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f8e40f-f7f2-4e41-b11b-2e552bbc12e4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:02.735096 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f8e40f-f7f2-4e41-b11b-2e552bbc12e4 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:02.742300 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 452/898] 10.222.0.66 () {56 vars in 1114 bytes} [Mon Jul 18 02:16:02 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:16:02.748739 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f8e40f-f7f2-4e41-b11b-2e552bbc12e4 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:02.752715 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8d91c3-2478-48bf-a0a2-b2a655590aee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:02.753932 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8d91c3-2478-48bf-a0a2-b2a655590aee None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:02.753932 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8d91c3-2478-48bf-a0a2-b2a655590aee None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:02.754236 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8d91c3-2478-48bf-a0a2-b2a655590aee None admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:02.755937 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8d91c3-2478-48bf-a0a2-b2a655590aee None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:16:02.755937 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8d91c3-2478-48bf-a0a2-b2a655590aee None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:02.756929 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8d91c3-2478-48bf-a0a2-b2a655590aee None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:02.770114 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 447/899] 10.222.0.66 () {56 vars in 1148 bytes} [Mon Jul 18 02:16:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:16:02.776609 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8d91c3-2478-48bf-a0a2-b2a655590aee None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:02.834942 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:02.866839 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f8e40f-f7f2-4e41-b11b-2e552bbc12e4 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:02.897397 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8d91c3-2478-48bf-a0a2-b2a655590aee None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:02.905292 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7665362-3036-41dd-b2b6-862f05719ea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:02.906063 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7665362-3036-41dd-b2b6-862f05719ea3 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:02.906395 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7665362-3036-41dd-b2b6-862f05719ea3 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:02.906678 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7665362-3036-41dd-b2b6-862f05719ea3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:02.907474 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7665362-3036-41dd-b2b6-862f05719ea3 None admin] RBAC: Adding query filter params (interface=public, service_id=17f999acb6ff46f388d81c4833731afe, region_id=RegionOne) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:16:02.907895 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7665362-3036-41dd-b2b6-862f05719ea3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:02.909445 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7665362-3036-41dd-b2b6-862f05719ea3 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:02.915130 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 453/900] 10.222.0.66 () {56 vars in 1251 bytes} [Mon Jul 18 02:16:02 2022] GET /identity/v3/endpoints?service_id=17f999acb6ff46f388d81c4833731afe&interface=public®ion_id=RegionOne => generated 194 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:16:02.921454 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7665362-3036-41dd-b2b6-862f05719ea3 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:02.939917 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d366bac-2da5-43e1-ad97-b298a210a545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:02.940761 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d366bac-2da5-43e1-ad97-b298a210a545 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:02.941047 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d366bac-2da5-43e1-ad97-b298a210a545 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:02.941321 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d366bac-2da5-43e1-ad97-b298a210a545 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:02.942475 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d366bac-2da5-43e1-ad97-b298a210a545 None admin] RBAC: Adding query filter params (role.id=a1b1364c0cb94959b01bd1c84faa1b9c, scope.project.id=76f7c0ae36ae49b09dcae4c3e9a875f4, user.id=768f872096584d4093fc616d547aabb6) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:16:02.942796 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d366bac-2da5-43e1-ad97-b298a210a545 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:02.944721 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d366bac-2da5-43e1-ad97-b298a210a545 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:02.953401 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 448/901] 10.222.0.66 () {56 vars in 1374 bytes} [Mon Jul 18 02:16:02 2022] GET /identity/v3/role_assignments?user.id=768f872096584d4093fc616d547aabb6&scope.project.id=76f7c0ae36ae49b09dcae4c3e9a875f4&role.id=a1b1364c0cb94959b01bd1c84faa1b9c => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:16:03.039939 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7665362-3036-41dd-b2b6-862f05719ea3 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:03.080970 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad18ba74-d958-4680-885d-b84b3d909c14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:03.081648 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad18ba74-d958-4680-885d-b84b3d909c14 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:03.081975 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad18ba74-d958-4680-885d-b84b3d909c14 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:03.082286 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad18ba74-d958-4680-885d-b84b3d909c14 None admin] PATH_INFO: `/v3/services` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:03.082966 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad18ba74-d958-4680-885d-b84b3d909c14 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:03.084353 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad18ba74-d958-4680-885d-b84b3d909c14 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:03.085644 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:03.092204 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 454/902] 10.222.0.66 () {56 vars in 1087 bytes} [Mon Jul 18 02:16:02 2022] GET /identity/v3/services => generated 1268 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 02:16:03.539242 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a0da14-c019-4cb2-b2bb-dd5601271558 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:03.539780 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a0da14-c019-4cb2-b2bb-dd5601271558 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:03.539780 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a0da14-c019-4cb2-b2bb-dd5601271558 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:03.541589 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 449/903] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:16:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:16:03.547216 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8849a61-5132-43a9-ad19-bc4d6bdc1647 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:03.547626 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8849a61-5132-43a9-ad19-bc4d6bdc1647 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:03.547796 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8849a61-5132-43a9-ad19-bc4d6bdc1647 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:03.565207 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-d8849a61-5132-43a9-ad19-bc4d6bdc1647 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:16:03.566491 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-d8849a61-5132-43a9-ad19-bc4d6bdc1647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:16:03.638919 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 455/904] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:16:03 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 02:16:03.661246 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a0da14-c019-4cb2-b2bb-dd5601271558 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:03.692275 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:03.769247 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a0da14-c019-4cb2-b2bb-dd5601271558 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:03.806725 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7655b31e-d6a0-4b59-8f25-e1b1c1eaf84a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:03.807465 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7655b31e-d6a0-4b59-8f25-e1b1c1eaf84a None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:03.807739 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7655b31e-d6a0-4b59-8f25-e1b1c1eaf84a None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:03.808004 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7655b31e-d6a0-4b59-8f25-e1b1c1eaf84a None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:03.808648 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7655b31e-d6a0-4b59-8f25-e1b1c1eaf84a None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:03.810015 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7655b31e-d6a0-4b59-8f25-e1b1c1eaf84a None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:03.822828 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-7655b31e-d6a0-4b59-8f25-e1b1c1eaf84a None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 18 02:16:03.822828 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:16:03.822828 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 02:16:03.822828 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 02:16:03.822828 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 02:16:03.822828 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 02:16:03.822828 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 02:16:03.822828 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 02:16:03.822828 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 18 02:16:03.822828 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:16:03.822828 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 02:16:03.822828 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:16:03.822828 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:16:03.822828 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:16:03.822828 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:16:03.822828 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:16:03.823832 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:16:03.823832 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:16:03.823832 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:16:03.823832 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:16:03.823832 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:16:03.823832 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:16:03.823832 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:16:03.823832 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 02:16:03.823832 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 02:16:03.823832 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 02:16:03.823832 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 02:16:03.823832 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:16:03.823832 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:16:03.823832 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:16:03.823832 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:16:03.823832 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:16:03.824696 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:16:03.824696 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:16:03.824696 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:16:03.824696 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:16:03.824696 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:16:03.824696 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:16:03.824696 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:16:03.824696 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:16:03.824696 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:16:03.824696 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:16:03.824696 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:16:03.824696 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 02:16:03.824696 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 02:16:03.824696 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 18 02:16:03.824696 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:16:03.824696 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 450/905] 10.222.0.66 () {56 vars in 1132 bytes} [Mon Jul 18 02:16:03 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:16:03.830229 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8849a61-5132-43a9-ad19-bc4d6bdc1647 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:03.932938 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8849a61-5132-43a9-ad19-bc4d6bdc1647 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:03.960220 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:03.968424 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bc3bf4-91d7-4aec-b16f-0c920fd65fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:03.969085 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bc3bf4-91d7-4aec-b16f-0c920fd65fc2 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:03.969364 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bc3bf4-91d7-4aec-b16f-0c920fd65fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:03.969634 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bc3bf4-91d7-4aec-b16f-0c920fd65fc2 None admin] PATH_INFO: `/v3/services` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:03.970359 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bc3bf4-91d7-4aec-b16f-0c920fd65fc2 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:16:03.970667 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bc3bf4-91d7-4aec-b16f-0c920fd65fc2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:03.972103 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bc3bf4-91d7-4aec-b16f-0c920fd65fc2 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:03.976951 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 456/906] 10.222.0.66 () {56 vars in 1126 bytes} [Mon Jul 18 02:16:03 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:16:03.983163 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7655b31e-d6a0-4b59-8f25-e1b1c1eaf84a None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:04.077037 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:04.086449 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7655b31e-d6a0-4b59-8f25-e1b1c1eaf84a None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:04.121197 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a32a337-afdc-461d-9908-78f156ad3471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:04.121916 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a32a337-afdc-461d-9908-78f156ad3471 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:04.122234 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a32a337-afdc-461d-9908-78f156ad3471 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:04.122514 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a32a337-afdc-461d-9908-78f156ad3471 None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:04.123272 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a32a337-afdc-461d-9908-78f156ad3471 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:16:04.123584 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a32a337-afdc-461d-9908-78f156ad3471 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:04.124966 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a32a337-afdc-461d-9908-78f156ad3471 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:04.130423 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 451/907] 10.222.0.66 () {56 vars in 1126 bytes} [Mon Jul 18 02:16:03 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:16:04.136761 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bc3bf4-91d7-4aec-b16f-0c920fd65fc2 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:04.244912 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bc3bf4-91d7-4aec-b16f-0c920fd65fc2 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:04.282735 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fbda24-7b0b-4c6f-92e1-a365e37de424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:04.283392 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fbda24-7b0b-4c6f-92e1-a365e37de424 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:04.283673 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fbda24-7b0b-4c6f-92e1-a365e37de424 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:04.283947 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fbda24-7b0b-4c6f-92e1-a365e37de424 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:04.284723 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fbda24-7b0b-4c6f-92e1-a365e37de424 None admin] RBAC: Adding query filter params (interface=public, service_id=5582e600c50b4860b2323b1d9270e367, region_id=RegionOne) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:16:04.285031 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fbda24-7b0b-4c6f-92e1-a365e37de424 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:04.286474 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fbda24-7b0b-4c6f-92e1-a365e37de424 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:04.292303 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 457/908] 10.222.0.66 () {56 vars in 1251 bytes} [Mon Jul 18 02:16:04 2022] GET /identity/v3/endpoints?service_id=5582e600c50b4860b2323b1d9270e367&interface=public®ion_id=RegionOne => generated 194 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 02:16:04.298051 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a32a337-afdc-461d-9908-78f156ad3471 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:04.407993 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a32a337-afdc-461d-9908-78f156ad3471 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:04.444794 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138f83b9-ac4a-416f-97c0-199d9e605487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:04.445457 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138f83b9-ac4a-416f-97c0-199d9e605487 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:04.445739 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138f83b9-ac4a-416f-97c0-199d9e605487 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:04.446012 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138f83b9-ac4a-416f-97c0-199d9e605487 None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:04.446731 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138f83b9-ac4a-416f-97c0-199d9e605487 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:04.448660 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138f83b9-ac4a-416f-97c0-199d9e605487 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:04.454887 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 452/909] 10.222.0.66 () {56 vars in 1087 bytes} [Mon Jul 18 02:16:04 2022] GET /identity/v3/services => generated 1268 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:16:04.741112 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09afa123-867d-40f5-a7d9-176ff7945cb8 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:04.741622 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09afa123-867d-40f5-a7d9-176ff7945cb8 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:04.741622 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09afa123-867d-40f5-a7d9-176ff7945cb8 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:04.743236 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 458/910] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:16:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:16:04.749151 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a161482-ed0a-445e-878f-b6a026a2f0ee None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:04.749594 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a161482-ed0a-445e-878f-b6a026a2f0ee None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:04.749787 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a161482-ed0a-445e-878f-b6a026a2f0ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:04.767724 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-3a161482-ed0a-445e-878f-b6a026a2f0ee None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:16:04.768919 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-3a161482-ed0a-445e-878f-b6a026a2f0ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:16:04.783784 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:04.844831 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 453/911] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:16:04 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 02:16:04.867170 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09afa123-867d-40f5-a7d9-176ff7945cb8 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:04.928057 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:04.976395 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09afa123-867d-40f5-a7d9-176ff7945cb8 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:05.012986 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c597515a-e788-456c-bfe1-9bff6c8b8b10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:05.013954 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c597515a-e788-456c-bfe1-9bff6c8b8b10 None admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:05.014404 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c597515a-e788-456c-bfe1-9bff6c8b8b10 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:05.015003 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c597515a-e788-456c-bfe1-9bff6c8b8b10 None admin] PATH_INFO: `/v3/services` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:05.015888 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c597515a-e788-456c-bfe1-9bff6c8b8b10 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:05.017493 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c597515a-e788-456c-bfe1-9bff6c8b8b10 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:05.085644 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 459/912] 10.222.0.66 () {60 vars in 1141 bytes} [Mon Jul 18 02:16:04 2022] POST /identity/v3/services => generated 241 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:16:05.166902 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39410b93-a4a7-48f7-8e21-5fdfb5d9ef9e None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:05.167169 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39410b93-a4a7-48f7-8e21-5fdfb5d9ef9e None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:05.167467 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39410b93-a4a7-48f7-8e21-5fdfb5d9ef9e None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:05.169274 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 454/913] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:16:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:16:05.175348 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9ad385-83e7-4a8f-adfb-3e5891b493dc None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:05.175633 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9ad385-83e7-4a8f-adfb-3e5891b493dc None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:05.175907 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9ad385-83e7-4a8f-adfb-3e5891b493dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:05.180655 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4225acde-7482-4b80-b765-4bc4f1cdf74c None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:05.180947 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4225acde-7482-4b80-b765-4bc4f1cdf74c None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:05.181224 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4225acde-7482-4b80-b765-4bc4f1cdf74c None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:05.183289 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 455/914] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:16:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 02:16:05.189565 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6dcdad-3856-4c3c-8a60-656838ba2af5 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:05.190253 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6dcdad-3856-4c3c-8a60-656838ba2af5 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:05.190752 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6dcdad-3856-4c3c-8a60-656838ba2af5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:05.198829 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-bc9ad385-83e7-4a8f-adfb-3e5891b493dc None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:16:05.200230 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-bc9ad385-83e7-4a8f-adfb-3e5891b493dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:16:05.213846 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-cb6dcdad-3856-4c3c-8a60-656838ba2af5 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:16:05.215568 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-cb6dcdad-3856-4c3c-8a60-656838ba2af5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:16:05.288403 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 460/915] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:16:05 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 02:16:05.302581 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 456/916] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:16:05 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 02:16:05.312619 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9ad385-83e7-4a8f-adfb-3e5891b493dc None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:05.326606 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6dcdad-3856-4c3c-8a60-656838ba2af5 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:05.362198 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:05.420078 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:05.435653 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9ad385-83e7-4a8f-adfb-3e5891b493dc None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:05.442935 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6dcdad-3856-4c3c-8a60-656838ba2af5 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:05.472337 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88158c3-3dd8-4334-a544-bfd0ae80f417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:05.473157 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88158c3-3dd8-4334-a544-bfd0ae80f417 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:05.473526 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88158c3-3dd8-4334-a544-bfd0ae80f417 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:05.473747 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88158c3-3dd8-4334-a544-bfd0ae80f417 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:05.480568 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845abcc5-c7d6-4312-9d26-b9c461b9b915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:05.481621 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845abcc5-c7d6-4312-9d26-b9c461b9b915 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:05.482106 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845abcc5-c7d6-4312-9d26-b9c461b9b915 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:05.482613 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845abcc5-c7d6-4312-9d26-b9c461b9b915 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:05.483440 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845abcc5-c7d6-4312-9d26-b9c461b9b915 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:05.484700 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88158c3-3dd8-4334-a544-bfd0ae80f417 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 02:16:05.485234 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845abcc5-c7d6-4312-9d26-b9c461b9b915 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:05.492973 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-a88158c3-3dd8-4334-a544-bfd0ae80f417 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 02:16:05.492973 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:16:05.492973 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:16:05.492973 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:16:05.492973 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:16:05.492973 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:16:05.492973 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:16:05.492973 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:16:05.492973 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:16:05.492973 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:16:05.492973 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:16:05.492973 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:16:05.492973 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 02:16:05.492973 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 02:16:05.492973 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 02:16:05.492973 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 02:16:05.494572 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 02:16:05.494572 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 02:16:05.494572 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 02:16:05.494572 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 02:16:05.494572 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:16:05.494572 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:16:05.494572 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:16:05.494572 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:16:05.494572 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:16:05.494572 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:16:05.494572 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:16:05.494572 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:16:05.494572 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:16:05.494572 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:16:05.494572 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:16:05.494572 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:16:05.495730 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:16:05.495730 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:16:05.495730 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:16:05.495730 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:16:05.495730 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 02:16:05.495730 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 02:16:05.495730 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 02:16:05.495730 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 02:16:05.495730 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 02:16:05.495730 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 02:16:05.495730 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 02:16:05.495730 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:16:05.501541 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 461/917] 10.222.0.66 () {56 vars in 1120 bytes} [Mon Jul 18 02:16:05 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:16:05.502918 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88158c3-3dd8-4334-a544-bfd0ae80f417 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:05.505177 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-845abcc5-c7d6-4312-9d26-b9c461b9b915 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 18 02:16:05.505177 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:16:05.505177 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 02:16:05.505177 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 02:16:05.505177 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 02:16:05.505177 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 02:16:05.505177 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 02:16:05.505177 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 02:16:05.505177 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 18 02:16:05.505177 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:16:05.505177 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 02:16:05.505177 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:16:05.505177 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:16:05.505177 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:16:05.505177 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:16:05.505177 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:16:05.505177 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:16:05.506207 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:16:05.506207 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:16:05.506207 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:16:05.506207 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:16:05.506207 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:16:05.506207 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:16:05.506207 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 02:16:05.506207 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 02:16:05.506207 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 02:16:05.506207 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 02:16:05.506207 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:16:05.506207 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:16:05.506207 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:16:05.506207 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:16:05.506207 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:16:05.506207 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:16:05.507455 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:16:05.507455 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:16:05.507455 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:16:05.507455 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:16:05.507455 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:16:05.507455 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:16:05.507455 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:16:05.507455 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:16:05.507455 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:16:05.507455 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:16:05.507455 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 02:16:05.507455 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 02:16:05.507455 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 18 02:16:05.507455 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:16:05.510174 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 457/918] 10.222.0.66 () {56 vars in 1114 bytes} [Mon Jul 18 02:16:05 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:16:05.513874 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845abcc5-c7d6-4312-9d26-b9c461b9b915 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:05.638240 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88158c3-3dd8-4334-a544-bfd0ae80f417 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:05.645888 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:05.648908 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845abcc5-c7d6-4312-9d26-b9c461b9b915 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:05.680574 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49057424-77aa-4d05-abf0-e49dc0bddc3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:05.681975 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49057424-77aa-4d05-abf0-e49dc0bddc3d None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:05.682489 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49057424-77aa-4d05-abf0-e49dc0bddc3d None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:05.682829 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49057424-77aa-4d05-abf0-e49dc0bddc3d None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:05.683678 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49057424-77aa-4d05-abf0-e49dc0bddc3d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:16:05.684201 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49057424-77aa-4d05-abf0-e49dc0bddc3d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:05.685821 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49057424-77aa-4d05-abf0-e49dc0bddc3d None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:05.699167 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 462/919] 10.222.0.66 () {56 vars in 1115 bytes} [Mon Jul 18 02:16:05 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 02:16:05.702615 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49057424-77aa-4d05-abf0-e49dc0bddc3d None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:05.702893 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fdc423-f64d-4dd1-b63b-7cd8d7b19fa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:05.703748 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fdc423-f64d-4dd1-b63b-7cd8d7b19fa6 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:05.704172 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fdc423-f64d-4dd1-b63b-7cd8d7b19fa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:05.704584 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fdc423-f64d-4dd1-b63b-7cd8d7b19fa6 None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:05.705435 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fdc423-f64d-4dd1-b63b-7cd8d7b19fa6 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:16:05.706124 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fdc423-f64d-4dd1-b63b-7cd8d7b19fa6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:05.708647 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fdc423-f64d-4dd1-b63b-7cd8d7b19fa6 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:05.715388 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 458/920] 10.222.0.66 () {56 vars in 1114 bytes} [Mon Jul 18 02:16:05 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:16:05.721740 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fdc423-f64d-4dd1-b63b-7cd8d7b19fa6 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:05.734129 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:05.836284 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49057424-77aa-4d05-abf0-e49dc0bddc3d None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:05.858518 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fdc423-f64d-4dd1-b63b-7cd8d7b19fa6 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:05.878870 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bc7ed6-a9f5-4740-934d-f56a6e6e7aeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:05.879749 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bc7ed6-a9f5-4740-934d-f56a6e6e7aeb None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:05.880032 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bc7ed6-a9f5-4740-934d-f56a6e6e7aeb None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:05.880306 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bc7ed6-a9f5-4740-934d-f56a6e6e7aeb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:05.893955 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bc7ed6-a9f5-4740-934d-f56a6e6e7aeb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:05.895907 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bc7ed6-a9f5-4740-934d-f56a6e6e7aeb None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:05.901476 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e66f28-a529-468b-ac46-3bbd7e943785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:05.902259 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e66f28-a529-468b-ac46-3bbd7e943785 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:05.902617 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e66f28-a529-468b-ac46-3bbd7e943785 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:05.902971 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e66f28-a529-468b-ac46-3bbd7e943785 None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:05.904269 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e66f28-a529-468b-ac46-3bbd7e943785 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:16:05.904652 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e66f28-a529-468b-ac46-3bbd7e943785 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:05.906976 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e66f28-a529-468b-ac46-3bbd7e943785 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:05.915482 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-a3bc7ed6-a9f5-4740-934d-f56a6e6e7aeb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:16:05.915482 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:16:05.915482 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:16:05.915482 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:16:05.915482 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:16:05.915482 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:16:05.915482 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:16:05.915482 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:16:05.915482 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:16:05.915482 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:16:05.915482 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:16:05.915482 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:16:05.915482 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 02:16:05.915482 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 02:16:05.915482 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 02:16:05.918354 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 02:16:05.918354 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:16:05.918354 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:16:05.918354 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:16:05.918354 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:16:05.918354 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:16:05.918354 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:16:05.918354 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:16:05.918354 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:16:05.918354 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:16:05.918354 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:16:05.918354 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:16:05.918354 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:16:05.918354 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:16:05.918354 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:16:05.918354 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:16:05.918354 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:16:05.920792 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 02:16:05.920792 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 02:16:05.920792 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:16:05.920792 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:16:05.921295 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 459/921] 10.222.0.66 () {56 vars in 1114 bytes} [Mon Jul 18 02:16:05 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:16:05.921295 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e66f28-a529-468b-ac46-3bbd7e943785 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:05.930106 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 463/922] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 18 02:16:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:16:05.930106 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bc7ed6-a9f5-4740-934d-f56a6e6e7aeb None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:05.948163 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:06.009104 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:06.050493 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bc7ed6-a9f5-4740-934d-f56a6e6e7aeb None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:06.050911 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e66f28-a529-468b-ac46-3bbd7e943785 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:06.094730 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55dcf9c-f8b4-48a1-b7dd-804aae149476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:06.095496 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55dcf9c-f8b4-48a1-b7dd-804aae149476 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:06.095721 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc3b17d-4a72-422c-bbb9-1c726d6522df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:06.095922 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55dcf9c-f8b4-48a1-b7dd-804aae149476 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:06.096315 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55dcf9c-f8b4-48a1-b7dd-804aae149476 None admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:06.096466 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc3b17d-4a72-422c-bbb9-1c726d6522df None admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:06.096648 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc3b17d-4a72-422c-bbb9-1c726d6522df None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:06.096925 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc3b17d-4a72-422c-bbb9-1c726d6522df None admin] PATH_INFO: `/v3/endpoints` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:06.097277 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55dcf9c-f8b4-48a1-b7dd-804aae149476 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:16:06.097652 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55dcf9c-f8b4-48a1-b7dd-804aae149476 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:06.097797 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc3b17d-4a72-422c-bbb9-1c726d6522df None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:06.099073 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc3b17d-4a72-422c-bbb9-1c726d6522df None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:06.099311 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55dcf9c-f8b4-48a1-b7dd-804aae149476 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:06.112506 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 464/923] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 18 02:16:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:16:06.118378 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55dcf9c-f8b4-48a1-b7dd-804aae149476 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:06.180748 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 460/924] 10.222.0.66 () {60 vars in 1144 bytes} [Mon Jul 18 02:16:05 2022] POST /identity/v3/endpoints => generated 346 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:16:06.210496 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:06.237178 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55dcf9c-f8b4-48a1-b7dd-804aae149476 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:06.282628 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e016606a-4cc2-4ee8-92fa-69f6135ad203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:06.283476 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e016606a-4cc2-4ee8-92fa-69f6135ad203 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:06.283753 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e016606a-4cc2-4ee8-92fa-69f6135ad203 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:06.284026 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e016606a-4cc2-4ee8-92fa-69f6135ad203 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:06.294284 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e016606a-4cc2-4ee8-92fa-69f6135ad203 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:06.296021 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e016606a-4cc2-4ee8-92fa-69f6135ad203 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:06.312415 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-e016606a-4cc2-4ee8-92fa-69f6135ad203 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 02:16:06.312415 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:16:06.312415 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:16:06.312415 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:16:06.312415 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:16:06.312415 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:16:06.312415 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:16:06.312415 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:16:06.312415 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:16:06.312415 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:16:06.312415 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:16:06.312415 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:16:06.312415 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 02:16:06.312415 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 02:16:06.312415 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 02:16:06.312415 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 02:16:06.313489 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:16:06.313489 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:16:06.313489 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 02:16:06.313489 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 02:16:06.313489 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 02:16:06.313489 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 02:16:06.313489 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:16:06.313489 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:16:06.313489 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:16:06.313489 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:16:06.313489 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:16:06.313489 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:16:06.313489 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:16:06.313489 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:16:06.313489 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:16:06.313489 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:16:06.313489 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:16:06.313489 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:16:06.314500 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:16:06.314500 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:16:06.314500 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 02:16:06.314500 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 02:16:06.314500 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 02:16:06.314500 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 02:16:06.314500 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 02:16:06.314500 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 02:16:06.314500 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 02:16:06.314500 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:16:06.318233 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 465/925] 10.222.0.66 () {56 vars in 1117 bytes} [Mon Jul 18 02:16:06 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 02:16:06.320018 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc3b17d-4a72-422c-bbb9-1c726d6522df None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:06.430685 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc3b17d-4a72-422c-bbb9-1c726d6522df None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:06.458187 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:06.467147 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f55085-0c72-4e1d-8848-f6b08f399a5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:06.467830 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f55085-0c72-4e1d-8848-f6b08f399a5d None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:06.468115 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f55085-0c72-4e1d-8848-f6b08f399a5d None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:06.468391 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f55085-0c72-4e1d-8848-f6b08f399a5d None admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:06.469225 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f55085-0c72-4e1d-8848-f6b08f399a5d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:16:06.469535 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f55085-0c72-4e1d-8848-f6b08f399a5d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:06.471014 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f55085-0c72-4e1d-8848-f6b08f399a5d None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:06.489307 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 461/926] 10.222.0.66 () {56 vars in 1149 bytes} [Mon Jul 18 02:16:06 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:16:06.495439 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e016606a-4cc2-4ee8-92fa-69f6135ad203 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:06.599697 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e016606a-4cc2-4ee8-92fa-69f6135ad203 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:06.626144 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108d1d24-ef7b-4592-8842-481e9a583c94 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:06.626438 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108d1d24-ef7b-4592-8842-481e9a583c94 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:06.626668 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108d1d24-ef7b-4592-8842-481e9a583c94 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:06.628165 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 462/927] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:16:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:16:06.635598 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:06.639718 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11d4029-5049-4dcf-90eb-5c1c2ed9447b None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:06.640253 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11d4029-5049-4dcf-90eb-5c1c2ed9447b None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:06.640685 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd8cd0d-693c-4a9a-b3c8-3a4a74bb5a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:06.640977 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11d4029-5049-4dcf-90eb-5c1c2ed9447b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:06.641475 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd8cd0d-693c-4a9a-b3c8-3a4a74bb5a4b None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:06.641753 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd8cd0d-693c-4a9a-b3c8-3a4a74bb5a4b None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:06.642025 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd8cd0d-693c-4a9a-b3c8-3a4a74bb5a4b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:06.653873 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd8cd0d-693c-4a9a-b3c8-3a4a74bb5a4b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:06.655414 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd8cd0d-693c-4a9a-b3c8-3a4a74bb5a4b None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:06.660131 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-f11d4029-5049-4dcf-90eb-5c1c2ed9447b None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:16:06.661521 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-f11d4029-5049-4dcf-90eb-5c1c2ed9447b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:16:06.674509 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-0dd8cd0d-693c-4a9a-b3c8-3a4a74bb5a4b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:16:06.674509 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:16:06.674509 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:16:06.674509 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:16:06.674509 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:16:06.674509 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:16:06.674509 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:16:06.674509 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:16:06.674509 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:16:06.674509 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:16:06.674509 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:16:06.674509 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:16:06.674509 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 02:16:06.674509 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 02:16:06.674509 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 02:16:06.675440 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 02:16:06.675440 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:16:06.675440 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:16:06.675440 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:16:06.675440 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:16:06.675440 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:16:06.675440 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:16:06.675440 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:16:06.675440 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:16:06.675440 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:16:06.675440 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:16:06.675440 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:16:06.675440 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:16:06.675440 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:16:06.675440 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:16:06.675440 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:16:06.675440 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:16:06.676343 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 02:16:06.676343 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 02:16:06.676343 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:16:06.676343 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:16:06.683308 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 466/928] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 18 02:16:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:16:06.684575 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd8cd0d-693c-4a9a-b3c8-3a4a74bb5a4b None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:06.743507 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 463/929] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:16:06 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 02:16:06.766743 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11d4029-5049-4dcf-90eb-5c1c2ed9447b None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:06.802124 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd8cd0d-693c-4a9a-b3c8-3a4a74bb5a4b None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:06.848059 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f47219c-35d7-4244-88ff-abfd1d1f8ff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:06.848699 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f47219c-35d7-4244-88ff-abfd1d1f8ff7 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:06.848986 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f47219c-35d7-4244-88ff-abfd1d1f8ff7 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:06.850597 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f47219c-35d7-4244-88ff-abfd1d1f8ff7 None admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:06.850597 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f47219c-35d7-4244-88ff-abfd1d1f8ff7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:16:06.850597 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f47219c-35d7-4244-88ff-abfd1d1f8ff7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:06.852098 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f47219c-35d7-4244-88ff-abfd1d1f8ff7 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:06.870192 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 467/930] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 18 02:16:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:16:06.875731 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f47219c-35d7-4244-88ff-abfd1d1f8ff7 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:06.885401 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11d4029-5049-4dcf-90eb-5c1c2ed9447b None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:06.906471 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:06.933708 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fe4548-66e0-45c8-a484-18862c586884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:06.934571 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fe4548-66e0-45c8-a484-18862c586884 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:06.935275 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:06.935736 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fe4548-66e0-45c8-a484-18862c586884 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:06.936104 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fe4548-66e0-45c8-a484-18862c586884 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:06.937136 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fe4548-66e0-45c8-a484-18862c586884 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:06.938841 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fe4548-66e0-45c8-a484-18862c586884 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:06.956194 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-12fe4548-66e0-45c8-a484-18862c586884 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 18 02:16:06.956194 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:16:06.956194 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 02:16:06.956194 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 02:16:06.956194 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 02:16:06.956194 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 02:16:06.956194 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 02:16:06.956194 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 02:16:06.956194 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 18 02:16:06.956194 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:16:06.956194 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 02:16:06.956194 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:16:06.956194 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:16:06.956194 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:16:06.956194 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:16:06.956194 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:16:06.958432 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:16:06.958432 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:16:06.958432 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:16:06.958432 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:16:06.958432 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:16:06.958432 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:16:06.958432 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:16:06.958432 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 02:16:06.958432 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 02:16:06.958432 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 02:16:06.958432 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 02:16:06.958432 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:16:06.958432 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:16:06.958432 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:16:06.958432 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:16:06.958432 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:16:06.959542 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:16:06.959542 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:16:06.959542 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:16:06.959542 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:16:06.959542 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:16:06.959542 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:16:06.959542 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:16:06.959542 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:16:06.959542 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:16:06.959542 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:16:06.959542 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:16:06.959542 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 02:16:06.959542 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 02:16:06.959542 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 18 02:16:06.959542 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:16:06.959542 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 464/931] 10.222.0.66 () {56 vars in 1132 bytes} [Mon Jul 18 02:16:06 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 02:16:06.966945 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fe4548-66e0-45c8-a484-18862c586884 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:07.005140 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f47219c-35d7-4244-88ff-abfd1d1f8ff7 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:07.053132 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4cfcaf-e6fa-4c2f-8732-f98efdf29cf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:07.054353 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4cfcaf-e6fa-4c2f-8732-f98efdf29cf1 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:07.054636 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4cfcaf-e6fa-4c2f-8732-f98efdf29cf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:07.054907 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4cfcaf-e6fa-4c2f-8732-f98efdf29cf1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:07.064394 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4cfcaf-e6fa-4c2f-8732-f98efdf29cf1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:07.066124 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4cfcaf-e6fa-4c2f-8732-f98efdf29cf1 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:07.081437 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-da4cfcaf-e6fa-4c2f-8732-f98efdf29cf1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 02:16:07.081437 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:16:07.081437 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:16:07.081437 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:16:07.081437 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:16:07.081437 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:16:07.081437 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:16:07.081437 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:16:07.081437 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:16:07.081437 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:16:07.081437 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:16:07.081437 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:16:07.081437 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 02:16:07.081437 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 02:16:07.081437 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 02:16:07.082488 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 02:16:07.082488 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:16:07.082488 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:16:07.082488 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:16:07.082488 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:16:07.082488 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:16:07.082488 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:16:07.082488 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:16:07.082488 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:16:07.082488 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:16:07.082488 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:16:07.082488 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:16:07.082488 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:16:07.082488 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:16:07.082488 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:16:07.082488 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:16:07.082488 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:16:07.083406 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 02:16:07.083406 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 02:16:07.083406 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 02:16:07.083406 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 02:16:07.083406 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 02:16:07.083406 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 02:16:07.083406 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 02:16:07.083406 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:16:07.083847 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fe4548-66e0-45c8-a484-18862c586884 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:07.085955 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 468/932] 10.222.0.66 () {56 vars in 1111 bytes} [Mon Jul 18 02:16:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:16:07.092429 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4cfcaf-e6fa-4c2f-8732-f98efdf29cf1 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:07.125126 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa02dac5-a441-4278-889d-9253a3222a49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:07.126166 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa02dac5-a441-4278-889d-9253a3222a49 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:07.126757 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa02dac5-a441-4278-889d-9253a3222a49 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:07.127314 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa02dac5-a441-4278-889d-9253a3222a49 None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:07.128220 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa02dac5-a441-4278-889d-9253a3222a49 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:16:07.128715 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa02dac5-a441-4278-889d-9253a3222a49 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:07.130568 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa02dac5-a441-4278-889d-9253a3222a49 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:07.141401 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 465/933] 10.222.0.66 () {56 vars in 1126 bytes} [Mon Jul 18 02:16:06 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 02:16:07.142516 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa02dac5-a441-4278-889d-9253a3222a49 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:07.191936 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:07.221309 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4cfcaf-e6fa-4c2f-8732-f98efdf29cf1 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:07.224169 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:07.256083 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa02dac5-a441-4278-889d-9253a3222a49 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:07.257728 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf017ed-5a0c-4de1-a351-4a5ec18de392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:07.258693 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf017ed-5a0c-4de1-a351-4a5ec18de392 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:07.259052 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf017ed-5a0c-4de1-a351-4a5ec18de392 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:07.259541 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf017ed-5a0c-4de1-a351-4a5ec18de392 None admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:07.260331 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf017ed-5a0c-4de1-a351-4a5ec18de392 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:16:07.260644 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf017ed-5a0c-4de1-a351-4a5ec18de392 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:07.262102 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf017ed-5a0c-4de1-a351-4a5ec18de392 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:07.279013 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 469/934] 10.222.0.66 () {56 vars in 1148 bytes} [Mon Jul 18 02:16:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:16:07.279013 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18539d65-a5d8-4bb4-99aa-0944fb10a0fe None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:07.279013 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18539d65-a5d8-4bb4-99aa-0944fb10a0fe None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:07.279013 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18539d65-a5d8-4bb4-99aa-0944fb10a0fe None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:07.279013 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 470/935] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:16:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:16:07.282242 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18539d65-a5d8-4bb4-99aa-0944fb10a0fe None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:07.296659 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567e683f-ba74-4796-acd8-a2ae187ef6a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:07.297343 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567e683f-ba74-4796-acd8-a2ae187ef6a9 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:07.297632 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567e683f-ba74-4796-acd8-a2ae187ef6a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:07.297911 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567e683f-ba74-4796-acd8-a2ae187ef6a9 None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:07.298673 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567e683f-ba74-4796-acd8-a2ae187ef6a9 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:16:07.298982 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567e683f-ba74-4796-acd8-a2ae187ef6a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:07.300437 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567e683f-ba74-4796-acd8-a2ae187ef6a9 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:07.305881 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 466/936] 10.222.0.66 () {56 vars in 1126 bytes} [Mon Jul 18 02:16:07 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:16:07.307229 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaef8ab-d795-4e16-9c06-9619d65a1675 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:07.307532 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaef8ab-d795-4e16-9c06-9619d65a1675 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:07.307817 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaef8ab-d795-4e16-9c06-9619d65a1675 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:07.324816 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-8aaef8ab-d795-4e16-9c06-9619d65a1675 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:16:07.326005 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-8aaef8ab-d795-4e16-9c06-9619d65a1675 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:16:07.398124 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 467/937] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:16:07 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 02:16:07.398124 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaef8ab-d795-4e16-9c06-9619d65a1675 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:07.421284 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18539d65-a5d8-4bb4-99aa-0944fb10a0fe None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:07.433393 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:07.462826 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f386e03-358a-4c33-8138-31724c4565e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:07.464025 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f386e03-358a-4c33-8138-31724c4565e1 None admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:07.464308 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f386e03-358a-4c33-8138-31724c4565e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:07.464579 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f386e03-358a-4c33-8138-31724c4565e1 None admin] PATH_INFO: `/v3/projects/76f7c0ae36ae49b09dcae4c3e9a875f4/users/768f872096584d4093fc616d547aabb6/roles/a1b1364c0cb94959b01bd1c84faa1b9c` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:07.466778 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f386e03-358a-4c33-8138-31724c4565e1 None admin] RBAC: Authorizing `identity:create_grant(project_id=76f7c0ae36ae49b09dcae4c3e9a875f4, user_id=768f872096584d4093fc616d547aabb6, role_id=a1b1364c0cb94959b01bd1c84faa1b9c)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:07.468201 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f386e03-358a-4c33-8138-31724c4565e1 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:07.503400 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaef8ab-d795-4e16-9c06-9619d65a1675 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:07.536981 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1969c789-21e4-4ce2-9a62-1feb14c28ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:07.537569 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1969c789-21e4-4ce2-9a62-1feb14c28ce3 None admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:07.537854 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1969c789-21e4-4ce2-9a62-1feb14c28ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:07.538166 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1969c789-21e4-4ce2-9a62-1feb14c28ce3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:07.538899 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1969c789-21e4-4ce2-9a62-1feb14c28ce3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:07.540983 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1969c789-21e4-4ce2-9a62-1feb14c28ce3 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:07.548454 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 471/938] 10.222.0.66 () {58 vars in 1439 bytes} [Mon Jul 18 02:16:07 2022] PUT /identity/v3/projects/76f7c0ae36ae49b09dcae4c3e9a875f4/users/768f872096584d4093fc616d547aabb6/roles/a1b1364c0cb94959b01bd1c84faa1b9c => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:16:07.549656 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f386e03-358a-4c33-8138-31724c4565e1 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:07.585666 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:07.639761 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 468/939] 10.222.0.66 () {60 vars in 1144 bytes} [Mon Jul 18 02:16:07 2022] POST /identity/v3/endpoints => generated 347 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:16:07.700478 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f386e03-358a-4c33-8138-31724c4565e1 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:07.747051 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403a6c4b-6eb5-48fa-8485-c0555d8c29ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:07.747877 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403a6c4b-6eb5-48fa-8485-c0555d8c29ce None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:07.748156 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403a6c4b-6eb5-48fa-8485-c0555d8c29ce None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:07.748428 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403a6c4b-6eb5-48fa-8485-c0555d8c29ce None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:07.749082 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403a6c4b-6eb5-48fa-8485-c0555d8c29ce None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:07.750555 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403a6c4b-6eb5-48fa-8485-c0555d8c29ce None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:07.763648 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-403a6c4b-6eb5-48fa-8485-c0555d8c29ce None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 02:16:07.763648 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:16:07.763648 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 02:16:07.763648 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 02:16:07.763648 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 02:16:07.763648 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 02:16:07.763648 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 02:16:07.763648 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 02:16:07.763648 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 02:16:07.763648 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:16:07.763648 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 02:16:07.763648 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:16:07.763648 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:16:07.763648 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:16:07.763648 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:16:07.763648 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:16:07.763648 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:16:07.764756 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:16:07.764756 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:16:07.764756 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:16:07.764756 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:16:07.764756 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:16:07.764756 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:16:07.764756 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 02:16:07.764756 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 02:16:07.764756 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 02:16:07.764756 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 02:16:07.764756 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:16:07.764756 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:16:07.764756 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:16:07.764756 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:16:07.764756 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:16:07.764756 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:16:07.765637 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:16:07.765637 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:16:07.765637 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:16:07.765637 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:16:07.765637 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:16:07.765637 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:16:07.765637 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:16:07.765637 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:16:07.765637 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:16:07.765637 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:16:07.765637 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 02:16:07.765637 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 02:16:07.765637 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 02:16:07.765637 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:16:07.765637 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 472/940] 10.222.0.66 () {56 vars in 1126 bytes} [Mon Jul 18 02:16:07 2022] GET /identity/v3/services/object-store => generated 93 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:16:07.773138 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1969c789-21e4-4ce2-9a62-1feb14c28ce3 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:07.879387 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:07.888526 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1969c789-21e4-4ce2-9a62-1feb14c28ce3 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:07.924312 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba148cb3-828b-4d21-a2d6-44d95e263f85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:07.925471 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba148cb3-828b-4d21-a2d6-44d95e263f85 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:07.926376 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba148cb3-828b-4d21-a2d6-44d95e263f85 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:07.926651 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba148cb3-828b-4d21-a2d6-44d95e263f85 None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:07.927369 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba148cb3-828b-4d21-a2d6-44d95e263f85 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:16:07.927687 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba148cb3-828b-4d21-a2d6-44d95e263f85 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:07.929089 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba148cb3-828b-4d21-a2d6-44d95e263f85 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:07.935042 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 469/941] 10.222.0.66 () {56 vars in 1122 bytes} [Mon Jul 18 02:16:07 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:16:07.940267 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403a6c4b-6eb5-48fa-8485-c0555d8c29ce None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:08.029728 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:08.046384 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403a6c4b-6eb5-48fa-8485-c0555d8c29ce None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:08.082639 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28974de9-8d0c-42b7-bc23-f157cecf52a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:08.083297 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28974de9-8d0c-42b7-bc23-f157cecf52a7 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:08.083576 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28974de9-8d0c-42b7-bc23-f157cecf52a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:08.083848 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28974de9-8d0c-42b7-bc23-f157cecf52a7 None admin] PATH_INFO: `/v3/services` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:08.084546 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28974de9-8d0c-42b7-bc23-f157cecf52a7 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:16:08.084850 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28974de9-8d0c-42b7-bc23-f157cecf52a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:08.086243 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28974de9-8d0c-42b7-bc23-f157cecf52a7 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:08.092097 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 473/942] 10.222.0.66 () {56 vars in 1122 bytes} [Mon Jul 18 02:16:07 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:16:08.097879 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba148cb3-828b-4d21-a2d6-44d95e263f85 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:08.205721 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba148cb3-828b-4d21-a2d6-44d95e263f85 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:08.241521 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3db712-c7e3-4e1c-be87-ae2fe5e06403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:08.242511 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3db712-c7e3-4e1c-be87-ae2fe5e06403 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:08.242972 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3db712-c7e3-4e1c-be87-ae2fe5e06403 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:08.244117 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3db712-c7e3-4e1c-be87-ae2fe5e06403 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:08.245086 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3db712-c7e3-4e1c-be87-ae2fe5e06403 None admin] RBAC: Adding query filter params (interface=public, service_id=e34e238b3ce0493bad850cc5706a270b, region_id=RegionOne) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:16:08.245574 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3db712-c7e3-4e1c-be87-ae2fe5e06403 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:08.247200 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3db712-c7e3-4e1c-be87-ae2fe5e06403 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:08.253293 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 470/943] 10.222.0.66 () {56 vars in 1251 bytes} [Mon Jul 18 02:16:08 2022] GET /identity/v3/endpoints?service_id=e34e238b3ce0493bad850cc5706a270b&interface=public®ion_id=RegionOne => generated 194 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:16:08.259352 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28974de9-8d0c-42b7-bc23-f157cecf52a7 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:08.351484 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae658b67-4088-4b83-b977-c071fdb2472f None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:08.351779 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae658b67-4088-4b83-b977-c071fdb2472f None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:08.352043 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae658b67-4088-4b83-b977-c071fdb2472f None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:08.354242 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 471/944] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:16:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:16:08.359180 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c46087-2655-49e5-8c4c-973ad0690e87 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:08.359468 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c46087-2655-49e5-8c4c-973ad0690e87 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:08.359747 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c46087-2655-49e5-8c4c-973ad0690e87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:08.367933 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28974de9-8d0c-42b7-bc23-f157cecf52a7 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:08.379341 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-d6c46087-2655-49e5-8c4c-973ad0690e87 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:16:08.380819 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-d6c46087-2655-49e5-8c4c-973ad0690e87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:16:08.396511 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:08.409798 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001adefa-31b5-4a73-b6d2-091dc6cb97ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:08.410509 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001adefa-31b5-4a73-b6d2-091dc6cb97ec None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:08.410821 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001adefa-31b5-4a73-b6d2-091dc6cb97ec None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:08.411103 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001adefa-31b5-4a73-b6d2-091dc6cb97ec None admin] PATH_INFO: `/v3/services` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:08.415956 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001adefa-31b5-4a73-b6d2-091dc6cb97ec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:08.415956 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001adefa-31b5-4a73-b6d2-091dc6cb97ec None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:08.458198 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 474/945] 10.222.0.66 () {56 vars in 1087 bytes} [Mon Jul 18 02:16:08 2022] GET /identity/v3/services => generated 1497 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 02:16:08.461296 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 472/946] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:16:08 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 02:16:08.484950 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001adefa-31b5-4a73-b6d2-091dc6cb97ec None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:08.502186 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:08.609240 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001adefa-31b5-4a73-b6d2-091dc6cb97ec None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:08.647782 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cc6322-bb49-4061-bce8-de3fdb4ff412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:08.648798 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cc6322-bb49-4061-bce8-de3fdb4ff412 None admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:08.649316 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cc6322-bb49-4061-bce8-de3fdb4ff412 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:08.650371 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cc6322-bb49-4061-bce8-de3fdb4ff412 None admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:08.651291 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cc6322-bb49-4061-bce8-de3fdb4ff412 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:08.652924 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cc6322-bb49-4061-bce8-de3fdb4ff412 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:08.654631 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: WARNING keystone.server.flask.common [None req-53cc6322-bb49-4061-bce8-de3fdb4ff412 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 18 02:16:08.712455 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.sql.core [None req-53cc6322-bb49-4061-bce8-de3fdb4ff412 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 18 02:16:08.712455 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 18 02:16:08.712455 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [parameters: {'id': 'd6813dc80ba045799ee2504e6c7ac2fc', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 18 02:16:08.712455 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80398) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 18 02:16:08.730507 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-53cc6322-bb49-4061-bce8-de3fdb4ff412 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id d6813dc80ba045799ee2504e6c7ac2fc.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id d6813dc80ba045799ee2504e6c7ac2fc. Jul 18 02:16:08.730507 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:16:08.730507 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 18 02:16:08.730507 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 18 02:16:08.730507 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 18 02:16:08.730507 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 18 02:16:08.730507 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 18 02:16:08.730507 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application result = self._query(query) Jul 18 02:16:08.730507 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 18 02:16:08.730507 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application conn.query(q) Jul 18 02:16:08.730507 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 18 02:16:08.730507 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 18 02:16:08.730507 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 18 02:16:08.731402 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application result.read() Jul 18 02:16:08.731402 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 18 02:16:08.731402 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 18 02:16:08.731402 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 18 02:16:08.731402 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 18 02:16:08.731402 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 18 02:16:08.731402 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 18 02:16:08.731402 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 18 02:16:08.731402 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 18 02:16:08.731402 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 18 02:16:08.731402 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:16:08.731402 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 18 02:16:08.731402 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:16:08.731402 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:16:08.731402 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 18 02:16:08.731402 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 18 02:16:08.731402 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 18 02:16:08.731402 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 18 02:16:08.732386 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 18 02:16:08.732386 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application next(self.gen) Jul 18 02:16:08.732386 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 18 02:16:08.732386 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application yield resource Jul 18 02:16:08.732386 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 18 02:16:08.732386 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application next(self.gen) Jul 18 02:16:08.732386 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 18 02:16:08.732386 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application self.session.rollback() Jul 18 02:16:08.732386 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 18 02:16:08.732386 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application self.force_reraise() Jul 18 02:16:08.732386 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 18 02:16:08.732386 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise self.value Jul 18 02:16:08.732386 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 18 02:16:08.732386 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 18 02:16:08.732386 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 18 02:16:08.732386 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application session.commit() Jul 18 02:16:08.732386 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 18 02:16:08.733286 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 18 02:16:08.733286 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 18 02:16:08.733286 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application self._prepare_impl() Jul 18 02:16:08.733286 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 18 02:16:08.733286 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application self.session.flush() Jul 18 02:16:08.733286 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 18 02:16:08.733286 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application self._flush(objects) Jul 18 02:16:08.733286 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 18 02:16:08.733286 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 18 02:16:08.733286 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 18 02:16:08.733286 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application compat.raise_( Jul 18 02:16:08.733286 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 18 02:16:08.733286 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception Jul 18 02:16:08.733286 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 18 02:16:08.733286 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application flush_context.execute() Jul 18 02:16:08.733286 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 18 02:16:08.733286 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rec.execute(self) Jul 18 02:16:08.734221 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 18 02:16:08.734221 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 18 02:16:08.734221 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 18 02:16:08.734221 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 18 02:16:08.734221 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 18 02:16:08.734221 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 18 02:16:08.734221 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 18 02:16:08.734221 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 18 02:16:08.734221 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 18 02:16:08.734221 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 18 02:16:08.734221 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 18 02:16:08.734221 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 18 02:16:08.734221 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 18 02:16:08.734221 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 18 02:16:08.734221 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 18 02:16:08.735054 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 18 02:16:08.735054 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 18 02:16:08.735054 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception Jul 18 02:16:08.735054 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 18 02:16:08.735054 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 18 02:16:08.735054 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 18 02:16:08.735054 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 18 02:16:08.735054 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 18 02:16:08.735054 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application result = self._query(query) Jul 18 02:16:08.735054 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 18 02:16:08.735054 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application conn.query(q) Jul 18 02:16:08.735054 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 18 02:16:08.735054 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 18 02:16:08.735054 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 18 02:16:08.735054 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application result.read() Jul 18 02:16:08.735054 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 18 02:16:08.735054 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 18 02:16:08.735944 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 18 02:16:08.735944 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 18 02:16:08.735944 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 18 02:16:08.735944 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 18 02:16:08.735944 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 18 02:16:08.735944 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 18 02:16:08.735944 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 18 02:16:08.735944 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 18 02:16:08.735944 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [parameters: {'id': 'd6813dc80ba045799ee2504e6c7ac2fc', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 18 02:16:08.735944 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 18 02:16:08.735944 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:16:08.735944 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 02:16:08.735944 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:16:08.735944 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:16:08.735944 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 18 02:16:08.736739 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 18 02:16:08.736739 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 18 02:16:08.736739 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 18 02:16:08.736739 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 18 02:16:08.736739 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:16:08.736739 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 02:16:08.736739 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:16:08.736739 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:16:08.736739 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:16:08.736739 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:16:08.736739 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:16:08.736739 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:16:08.736739 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:16:08.736739 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:16:08.736739 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:16:08.736739 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:16:08.736739 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:16:08.737668 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:16:08.737668 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 18 02:16:08.737668 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 18 02:16:08.737668 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:16:08.737668 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:16:08.737668 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 18 02:16:08.737668 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 18 02:16:08.737668 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id d6813dc80ba045799ee2504e6c7ac2fc. Jul 18 02:16:08.737668 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:16:08.741491 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 475/947] 10.222.0.66 () {60 vars in 1140 bytes} [Mon Jul 18 02:16:08 2022] POST /identity/v3/projects => generated 262 bytes in 257 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:16:08.748410 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c46087-2655-49e5-8c4c-973ad0690e87 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:08.856796 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c46087-2655-49e5-8c4c-973ad0690e87 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:08.893038 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c708d7-1151-4d02-8507-b14dcd46f230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:08.894700 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c708d7-1151-4d02-8507-b14dcd46f230 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:08.895214 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c708d7-1151-4d02-8507-b14dcd46f230 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:08.895710 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c708d7-1151-4d02-8507-b14dcd46f230 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:08.903674 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c708d7-1151-4d02-8507-b14dcd46f230 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:08.905375 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c708d7-1151-4d02-8507-b14dcd46f230 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:08.920777 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-26c708d7-1151-4d02-8507-b14dcd46f230 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 02:16:08.920777 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:16:08.920777 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:16:08.920777 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:16:08.920777 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:16:08.920777 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:16:08.920777 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:16:08.920777 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:16:08.920777 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:16:08.920777 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:16:08.920777 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:16:08.920777 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:16:08.920777 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 02:16:08.920777 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 02:16:08.920777 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 02:16:08.921767 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 02:16:08.921767 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:16:08.921767 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:16:08.921767 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:16:08.921767 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:16:08.921767 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:16:08.921767 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:16:08.921767 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:16:08.921767 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:16:08.921767 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:16:08.921767 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:16:08.921767 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:16:08.921767 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:16:08.921767 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:16:08.921767 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:16:08.921767 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:16:08.921767 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:16:08.922713 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 02:16:08.922713 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 02:16:08.922713 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 02:16:08.922713 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 02:16:08.922713 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 02:16:08.922713 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 02:16:08.922713 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 02:16:08.922713 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:16:08.925432 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 473/948] 10.222.0.66 () {56 vars in 1111 bytes} [Mon Jul 18 02:16:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:16:08.932136 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cc6322-bb49-4061-bce8-de3fdb4ff412 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:09.004239 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:09.039574 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cc6322-bb49-4061-bce8-de3fdb4ff412 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:09.076441 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf59c49a-e7ad-4949-89db-e831f2ef02d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:09.077197 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf59c49a-e7ad-4949-89db-e831f2ef02d7 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:09.077481 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf59c49a-e7ad-4949-89db-e831f2ef02d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:09.077755 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf59c49a-e7ad-4949-89db-e831f2ef02d7 None admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:09.078611 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf59c49a-e7ad-4949-89db-e831f2ef02d7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:16:09.078923 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf59c49a-e7ad-4949-89db-e831f2ef02d7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:09.080589 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf59c49a-e7ad-4949-89db-e831f2ef02d7 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:09.103283 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 476/949] 10.222.0.66 () {56 vars in 1112 bytes} [Mon Jul 18 02:16:08 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:16:09.706048 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029b51eb-87af-4f77-be86-646c25b7d2cd None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:09.706412 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029b51eb-87af-4f77-be86-646c25b7d2cd None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:09.706645 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029b51eb-87af-4f77-be86-646c25b7d2cd None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:09.708294 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 474/950] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:16:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:16:09.713928 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7fc3ba-d812-46d5-98b6-ae31164ee589 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:09.714260 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7fc3ba-d812-46d5-98b6-ae31164ee589 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:09.714545 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7fc3ba-d812-46d5-98b6-ae31164ee589 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:09.732484 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-8e7fc3ba-d812-46d5-98b6-ae31164ee589 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:16:09.734018 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-8e7fc3ba-d812-46d5-98b6-ae31164ee589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:16:09.805444 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 477/951] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:16:09 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 02:16:09.829118 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029b51eb-87af-4f77-be86-646c25b7d2cd None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:09.842532 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:09.850071 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c280ebac-03d8-44d7-80c4-0f8e29e505ad None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:09.851169 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c280ebac-03d8-44d7-80c4-0f8e29e505ad None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:09.851169 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c280ebac-03d8-44d7-80c4-0f8e29e505ad None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:09.852538 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 478/952] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:16:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:16:09.858535 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937345f9-74e4-47f8-84f9-e68c52022bc4 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:09.858976 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937345f9-74e4-47f8-84f9-e68c52022bc4 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:09.859131 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937345f9-74e4-47f8-84f9-e68c52022bc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:09.882237 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-937345f9-74e4-47f8-84f9-e68c52022bc4 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:16:09.883523 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-937345f9-74e4-47f8-84f9-e68c52022bc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:16:09.957431 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:09.959487 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029b51eb-87af-4f77-be86-646c25b7d2cd None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:09.981093 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 479/953] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:16:09 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 02:16:10.001946 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937345f9-74e4-47f8-84f9-e68c52022bc4 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:10.011669 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08028223-b259-4460-8b2a-8914c07d031e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:10.012717 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08028223-b259-4460-8b2a-8914c07d031e None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:10.013608 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08028223-b259-4460-8b2a-8914c07d031e None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:10.014062 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08028223-b259-4460-8b2a-8914c07d031e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:10.029304 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08028223-b259-4460-8b2a-8914c07d031e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 02:16:10.042883 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-08028223-b259-4460-8b2a-8914c07d031e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 02:16:10.042883 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:16:10.042883 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:16:10.042883 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:16:10.042883 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:16:10.042883 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:16:10.042883 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:16:10.042883 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:16:10.042883 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:16:10.042883 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:16:10.042883 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:16:10.042883 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:16:10.042883 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 02:16:10.042883 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 02:16:10.042883 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 02:16:10.042883 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 02:16:10.044187 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 02:16:10.044187 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 02:16:10.044187 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 02:16:10.044187 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 02:16:10.044187 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:16:10.044187 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:16:10.044187 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:16:10.044187 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:16:10.044187 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:16:10.044187 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:16:10.044187 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:16:10.044187 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:16:10.044187 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:16:10.044187 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:16:10.044187 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:16:10.044187 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:16:10.045609 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:16:10.045609 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:16:10.045609 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:16:10.045609 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:16:10.045609 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 02:16:10.045609 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 02:16:10.045609 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 02:16:10.045609 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 02:16:10.045609 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 02:16:10.045609 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 02:16:10.045609 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 02:16:10.045609 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:16:10.048099 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 475/954] 10.222.0.66 () {56 vars in 1120 bytes} [Mon Jul 18 02:16:09 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:16:10.054115 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08028223-b259-4460-8b2a-8914c07d031e None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:10.115050 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937345f9-74e4-47f8-84f9-e68c52022bc4 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:10.131078 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:10.151948 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d95a37-7503-4bce-8f98-ac564429c2e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:10.152676 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d95a37-7503-4bce-8f98-ac564429c2e1 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:10.152984 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d95a37-7503-4bce-8f98-ac564429c2e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:10.153338 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d95a37-7503-4bce-8f98-ac564429c2e1 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:10.154120 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d95a37-7503-4bce-8f98-ac564429c2e1 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:10.155521 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d95a37-7503-4bce-8f98-ac564429c2e1 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:10.168548 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-54d95a37-7503-4bce-8f98-ac564429c2e1 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 18 02:16:10.168548 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:16:10.168548 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 02:16:10.168548 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 02:16:10.168548 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 02:16:10.168548 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 02:16:10.168548 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 02:16:10.168548 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 02:16:10.168548 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 18 02:16:10.168548 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:16:10.168548 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 02:16:10.168548 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:16:10.168548 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:16:10.168548 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:16:10.168548 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:16:10.168548 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:16:10.168548 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:16:10.169731 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:16:10.169731 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:16:10.169731 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:16:10.169731 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:16:10.169731 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:16:10.169731 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:16:10.169731 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 02:16:10.169731 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 02:16:10.169731 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 02:16:10.169731 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 02:16:10.169731 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:16:10.169731 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:16:10.169731 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:16:10.169731 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:16:10.169731 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:16:10.169731 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:16:10.170923 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:16:10.170923 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:16:10.170923 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:16:10.170923 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:16:10.170923 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:16:10.170923 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:16:10.170923 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:16:10.170923 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:16:10.170923 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:16:10.170923 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:16:10.170923 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 02:16:10.170923 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 02:16:10.170923 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 18 02:16:10.170923 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:16:10.173968 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08028223-b259-4460-8b2a-8914c07d031e None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:10.174205 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 480/955] 10.222.0.66 () {56 vars in 1111 bytes} [Mon Jul 18 02:16:10 2022] GET /identity/v3/services/compute => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 02:16:10.176295 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d95a37-7503-4bce-8f98-ac564429c2e1 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:10.211855 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40e6bc7-b24d-4941-867f-97cedda7b2d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:10.213469 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40e6bc7-b24d-4941-867f-97cedda7b2d9 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:10.213971 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40e6bc7-b24d-4941-867f-97cedda7b2d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:10.214492 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40e6bc7-b24d-4941-867f-97cedda7b2d9 None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:10.215450 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40e6bc7-b24d-4941-867f-97cedda7b2d9 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:16:10.215978 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40e6bc7-b24d-4941-867f-97cedda7b2d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:10.217640 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40e6bc7-b24d-4941-867f-97cedda7b2d9 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:10.230137 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 476/956] 10.222.0.66 () {56 vars in 1115 bytes} [Mon Jul 18 02:16:10 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 02:16:10.233080 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40e6bc7-b24d-4941-867f-97cedda7b2d9 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:10.235143 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:10.293122 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d95a37-7503-4bce-8f98-ac564429c2e1 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:10.340912 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10620362-c48f-4c03-b41c-db6b902fbde5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:10.342237 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10620362-c48f-4c03-b41c-db6b902fbde5 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:10.343084 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10620362-c48f-4c03-b41c-db6b902fbde5 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:10.343963 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10620362-c48f-4c03-b41c-db6b902fbde5 None admin] PATH_INFO: `/v3/services` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:10.344902 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10620362-c48f-4c03-b41c-db6b902fbde5 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:16:10.345442 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10620362-c48f-4c03-b41c-db6b902fbde5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:10.347614 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10620362-c48f-4c03-b41c-db6b902fbde5 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:10.358134 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 481/957] 10.222.0.66 () {56 vars in 1112 bytes} [Mon Jul 18 02:16:10 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 02:16:10.364662 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10620362-c48f-4c03-b41c-db6b902fbde5 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:10.370355 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40e6bc7-b24d-4941-867f-97cedda7b2d9 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:10.402503 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:10.407881 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0807a25c-9d03-4d05-9578-6955baf02a3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:10.413950 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0807a25c-9d03-4d05-9578-6955baf02a3c None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:10.413950 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0807a25c-9d03-4d05-9578-6955baf02a3c None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:10.413950 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0807a25c-9d03-4d05-9578-6955baf02a3c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:10.421885 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0807a25c-9d03-4d05-9578-6955baf02a3c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:10.423814 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0807a25c-9d03-4d05-9578-6955baf02a3c None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:10.442119 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-0807a25c-9d03-4d05-9578-6955baf02a3c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:16:10.442119 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:16:10.442119 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:16:10.442119 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:16:10.442119 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:16:10.442119 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:16:10.442119 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:16:10.442119 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:16:10.442119 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:16:10.442119 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:16:10.442119 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:16:10.442119 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:16:10.442119 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 02:16:10.442119 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 02:16:10.442119 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 02:16:10.443140 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 02:16:10.443140 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:16:10.443140 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:16:10.443140 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:16:10.443140 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:16:10.443140 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:16:10.443140 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:16:10.443140 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:16:10.443140 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:16:10.443140 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:16:10.443140 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:16:10.443140 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:16:10.443140 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:16:10.443140 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:16:10.443140 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:16:10.443140 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:16:10.443140 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:16:10.444082 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 02:16:10.444082 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 02:16:10.444082 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:16:10.444082 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:16:10.446238 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 477/958] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 18 02:16:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 02:16:10.452099 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0807a25c-9d03-4d05-9578-6955baf02a3c None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:10.493981 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10620362-c48f-4c03-b41c-db6b902fbde5 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:10.512724 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:10.532756 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8515d97c-2aca-4ed3-9cc2-ca889d63ac99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:10.533612 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8515d97c-2aca-4ed3-9cc2-ca889d63ac99 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:10.533997 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8515d97c-2aca-4ed3-9cc2-ca889d63ac99 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:10.534406 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8515d97c-2aca-4ed3-9cc2-ca889d63ac99 None admin] PATH_INFO: `/v3/services` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:10.535635 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8515d97c-2aca-4ed3-9cc2-ca889d63ac99 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:16:10.536036 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8515d97c-2aca-4ed3-9cc2-ca889d63ac99 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:10.537552 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8515d97c-2aca-4ed3-9cc2-ca889d63ac99 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:10.543847 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 482/959] 10.222.0.66 () {56 vars in 1112 bytes} [Mon Jul 18 02:16:10 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 02:16:10.588839 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0807a25c-9d03-4d05-9578-6955baf02a3c None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:10.594072 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9f1d63-b421-4540-a953-09c6c51911d4 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:10.594393 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9f1d63-b421-4540-a953-09c6c51911d4 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:10.594679 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9f1d63-b421-4540-a953-09c6c51911d4 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:10.596436 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 483/960] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:16:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 02:16:10.602013 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b03bb18-8883-4784-9763-f24f660dabf1 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:10.602324 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b03bb18-8883-4784-9763-f24f660dabf1 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:10.602682 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b03bb18-8883-4784-9763-f24f660dabf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:10.633472 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e67893-7f28-4c8e-8d52-042cb56eef6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:10.635236 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e67893-7f28-4c8e-8d52-042cb56eef6f None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:10.635524 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e67893-7f28-4c8e-8d52-042cb56eef6f None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:10.635801 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e67893-7f28-4c8e-8d52-042cb56eef6f None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:10.636555 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e67893-7f28-4c8e-8d52-042cb56eef6f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:16:10.636865 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e67893-7f28-4c8e-8d52-042cb56eef6f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:10.638363 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e67893-7f28-4c8e-8d52-042cb56eef6f None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:10.639520 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-1b03bb18-8883-4784-9763-f24f660dabf1 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:16:10.641379 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-1b03bb18-8883-4784-9763-f24f660dabf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:16:10.653056 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 478/961] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 18 02:16:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 02:16:10.659016 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e67893-7f28-4c8e-8d52-042cb56eef6f None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:10.711497 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:10.734989 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 484/962] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:16:10 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 02:16:10.757910 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b03bb18-8883-4784-9763-f24f660dabf1 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:10.785212 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e67893-7f28-4c8e-8d52-042cb56eef6f None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:10.832131 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6f3a5e-458d-4fc0-bed2-dc2635010e5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:10.833504 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6f3a5e-458d-4fc0-bed2-dc2635010e5c None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:10.834196 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6f3a5e-458d-4fc0-bed2-dc2635010e5c None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:10.835137 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6f3a5e-458d-4fc0-bed2-dc2635010e5c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:10.846306 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6f3a5e-458d-4fc0-bed2-dc2635010e5c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:10.848034 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6f3a5e-458d-4fc0-bed2-dc2635010e5c None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:10.869744 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-8d6f3a5e-458d-4fc0-bed2-dc2635010e5c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 02:16:10.869744 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:16:10.869744 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:16:10.869744 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:16:10.869744 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:16:10.869744 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:16:10.869744 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:16:10.869744 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:16:10.869744 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:16:10.869744 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:16:10.869744 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:16:10.869744 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:16:10.869744 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 02:16:10.869744 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 02:16:10.869744 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 02:16:10.869744 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 02:16:10.871054 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:16:10.871054 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:16:10.871054 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 02:16:10.871054 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 02:16:10.871054 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 02:16:10.871054 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 02:16:10.871054 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:16:10.871054 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:16:10.871054 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:16:10.871054 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:16:10.871054 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:16:10.871054 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:16:10.871054 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:16:10.871054 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:16:10.871054 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:16:10.871054 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:16:10.871054 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:16:10.871054 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:16:10.872075 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:16:10.872075 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:16:10.872075 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 02:16:10.872075 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 02:16:10.872075 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 02:16:10.872075 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 02:16:10.872075 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 02:16:10.872075 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 02:16:10.872075 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 02:16:10.872075 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:16:10.872075 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 479/963] 10.222.0.66 () {56 vars in 1117 bytes} [Mon Jul 18 02:16:10 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 02:16:10.876646 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6f3a5e-458d-4fc0-bed2-dc2635010e5c None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:10.890522 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b03bb18-8883-4784-9763-f24f660dabf1 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:10.905662 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:10.935857 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b66271-4c47-4b2f-8426-2e8f26780a6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:10.936844 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b66271-4c47-4b2f-8426-2e8f26780a6c None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:10.937202 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b66271-4c47-4b2f-8426-2e8f26780a6c None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:10.937601 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b66271-4c47-4b2f-8426-2e8f26780a6c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:10.938392 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b66271-4c47-4b2f-8426-2e8f26780a6c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:10.939834 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b66271-4c47-4b2f-8426-2e8f26780a6c None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:10.953810 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-d2b66271-4c47-4b2f-8426-2e8f26780a6c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 02:16:10.953810 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:16:10.953810 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 02:16:10.953810 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 02:16:10.953810 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 02:16:10.953810 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 02:16:10.953810 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 02:16:10.953810 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 02:16:10.953810 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 02:16:10.953810 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:16:10.953810 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 02:16:10.953810 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:16:10.953810 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:16:10.953810 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:16:10.953810 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:16:10.953810 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:16:10.953810 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:16:10.955746 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:16:10.955746 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:16:10.955746 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:16:10.955746 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:16:10.955746 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:16:10.955746 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:16:10.955746 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 02:16:10.955746 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 02:16:10.955746 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 02:16:10.955746 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 02:16:10.955746 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:16:10.955746 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:16:10.955746 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:16:10.955746 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:16:10.955746 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:16:10.955746 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:16:10.956896 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:16:10.956896 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:16:10.956896 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:16:10.956896 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:16:10.956896 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:16:10.956896 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:16:10.956896 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:16:10.956896 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:16:10.956896 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:16:10.956896 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:16:10.956896 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 02:16:10.956896 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 02:16:10.956896 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 02:16:10.956896 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:16:10.958650 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 485/964] 10.222.0.66 () {56 vars in 1126 bytes} [Mon Jul 18 02:16:10 2022] GET /identity/v3/services/object-store => generated 93 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 02:16:10.962968 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b66271-4c47-4b2f-8426-2e8f26780a6c None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:11.015411 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6f3a5e-458d-4fc0-bed2-dc2635010e5c None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:11.019401 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:11.069901 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fefed5-08ee-40c2-b1b8-bef597f919ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:11.071141 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fefed5-08ee-40c2-b1b8-bef597f919ea None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:11.072068 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fefed5-08ee-40c2-b1b8-bef597f919ea None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:11.072647 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fefed5-08ee-40c2-b1b8-bef597f919ea None admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:11.073770 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fefed5-08ee-40c2-b1b8-bef597f919ea None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:16:11.074585 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fefed5-08ee-40c2-b1b8-bef597f919ea None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:11.076859 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fefed5-08ee-40c2-b1b8-bef597f919ea None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:11.077987 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b66271-4c47-4b2f-8426-2e8f26780a6c None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:11.102425 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 480/965] 10.222.0.66 () {56 vars in 1149 bytes} [Mon Jul 18 02:16:10 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:16:11.108548 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fefed5-08ee-40c2-b1b8-bef597f919ea None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:11.121538 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be2ddc5-2750-412c-9b21-8615cba8423b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:11.123232 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be2ddc5-2750-412c-9b21-8615cba8423b None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:11.123794 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be2ddc5-2750-412c-9b21-8615cba8423b None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:11.124260 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be2ddc5-2750-412c-9b21-8615cba8423b None admin] PATH_INFO: `/v3/services` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:11.125276 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be2ddc5-2750-412c-9b21-8615cba8423b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:16:11.125745 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be2ddc5-2750-412c-9b21-8615cba8423b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:11.127975 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be2ddc5-2750-412c-9b21-8615cba8423b None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:11.135192 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 486/966] 10.222.0.66 () {56 vars in 1122 bytes} [Mon Jul 18 02:16:10 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 02:16:11.140714 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be2ddc5-2750-412c-9b21-8615cba8423b None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:11.191969 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:11.246717 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fefed5-08ee-40c2-b1b8-bef597f919ea None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:11.259348 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be2ddc5-2750-412c-9b21-8615cba8423b None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:11.283788 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dca9b3e-db8b-40f3-b16b-d3df49cccb18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:11.284440 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dca9b3e-db8b-40f3-b16b-d3df49cccb18 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:11.284723 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dca9b3e-db8b-40f3-b16b-d3df49cccb18 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:11.285020 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dca9b3e-db8b-40f3-b16b-d3df49cccb18 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:11.295481 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b48b7e-57f3-4e1c-928b-9da5d9da2a06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:11.296149 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b48b7e-57f3-4e1c-928b-9da5d9da2a06 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:11.296432 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b48b7e-57f3-4e1c-928b-9da5d9da2a06 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:11.296707 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b48b7e-57f3-4e1c-928b-9da5d9da2a06 None admin] PATH_INFO: `/v3/services` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:11.297405 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b48b7e-57f3-4e1c-928b-9da5d9da2a06 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:16:11.297713 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b48b7e-57f3-4e1c-928b-9da5d9da2a06 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:11.299117 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b48b7e-57f3-4e1c-928b-9da5d9da2a06 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:11.309038 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 487/967] 10.222.0.66 () {56 vars in 1122 bytes} [Mon Jul 18 02:16:11 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 02:16:11.309038 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d1ca7e-99d2-493a-ae4d-8a6a09cf3403 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:11.309038 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d1ca7e-99d2-493a-ae4d-8a6a09cf3403 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:11.309038 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d1ca7e-99d2-493a-ae4d-8a6a09cf3403 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:11.311736 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 488/968] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:16:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:16:11.312499 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d1ca7e-99d2-493a-ae4d-8a6a09cf3403 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:11.324090 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:11.366175 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dca9b3e-db8b-40f3-b16b-d3df49cccb18 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:11.368072 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dca9b3e-db8b-40f3-b16b-d3df49cccb18 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:11.419779 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d1ca7e-99d2-493a-ae4d-8a6a09cf3403 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:11.421535 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:11.457573 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d74780-c0f8-4bdd-b126-1fae5df7abf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:11.458342 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d74780-c0f8-4bdd-b126-1fae5df7abf4 None admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:11.458635 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d74780-c0f8-4bdd-b126-1fae5df7abf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:11.458914 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d74780-c0f8-4bdd-b126-1fae5df7abf4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:11.459657 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d74780-c0f8-4bdd-b126-1fae5df7abf4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:11.461039 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d74780-c0f8-4bdd-b126-1fae5df7abf4 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:11.476480 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 481/969] 10.222.0.66 () {58 vars in 1282 bytes} [Mon Jul 18 02:16:11 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 18 02:16:11.478190 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2d113b-422b-4a66-bd15-344b8422f3bd None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:11.478516 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2d113b-422b-4a66-bd15-344b8422f3bd None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:11.478796 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2d113b-422b-4a66-bd15-344b8422f3bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:11.495199 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-0f2d113b-422b-4a66-bd15-344b8422f3bd None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:16:11.496365 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-0f2d113b-422b-4a66-bd15-344b8422f3bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:16:11.547516 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 489/970] 10.222.0.66 () {60 vars in 1144 bytes} [Mon Jul 18 02:16:11 2022] POST /identity/v3/endpoints => generated 349 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:16:11.548748 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d74780-c0f8-4bdd-b126-1fae5df7abf4 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:11.558855 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:11.590836 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 482/971] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:16:11 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 02:16:11.611571 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2d113b-422b-4a66-bd15-344b8422f3bd None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:11.699622 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d74780-c0f8-4bdd-b126-1fae5df7abf4 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:11.740488 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2d113b-422b-4a66-bd15-344b8422f3bd None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:11.744953 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28431b7-4ff3-472e-a905-f3c837b8f7a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:11.745130 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28431b7-4ff3-472e-a905-f3c837b8f7a7 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:11.745372 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28431b7-4ff3-472e-a905-f3c837b8f7a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:11.745677 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28431b7-4ff3-472e-a905-f3c837b8f7a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:11.757308 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28431b7-4ff3-472e-a905-f3c837b8f7a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:11.759063 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28431b7-4ff3-472e-a905-f3c837b8f7a7 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:11.782411 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063ff975-4db8-44c5-9d2d-e298d90b8521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:11.782839 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-f28431b7-4ff3-472e-a905-f3c837b8f7a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:16:11.782839 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:16:11.782839 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:16:11.782839 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:16:11.782839 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:16:11.782839 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:16:11.782839 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:16:11.782839 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:16:11.782839 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:16:11.782839 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:16:11.782839 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:16:11.782839 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:16:11.782839 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 02:16:11.782839 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 02:16:11.782839 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 02:16:11.783720 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 02:16:11.783720 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:16:11.783720 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:16:11.783720 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:16:11.783720 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:16:11.783720 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:16:11.783720 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:16:11.783720 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:16:11.783720 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:16:11.783720 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:16:11.783720 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:16:11.783720 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:16:11.783720 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:16:11.783720 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:16:11.783720 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:16:11.783720 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:16:11.783720 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:16:11.784660 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 02:16:11.784660 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 02:16:11.784660 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:16:11.784660 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:16:11.784885 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063ff975-4db8-44c5-9d2d-e298d90b8521 None admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:11.784885 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063ff975-4db8-44c5-9d2d-e298d90b8521 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:11.784885 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063ff975-4db8-44c5-9d2d-e298d90b8521 None admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:11.784885 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063ff975-4db8-44c5-9d2d-e298d90b8521 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:11.785724 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063ff975-4db8-44c5-9d2d-e298d90b8521 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:11.787120 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: WARNING keystone.server.flask.common [None req-063ff975-4db8-44c5-9d2d-e298d90b8521 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 18 02:16:11.787752 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 490/972] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 18 02:16:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:16:11.794481 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28431b7-4ff3-472e-a905-f3c837b8f7a7 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:11.808684 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:11.828280 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.sql.core [None req-063ff975-4db8-44c5-9d2d-e298d90b8521 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 18 02:16:11.828280 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 18 02:16:11.828280 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [parameters: {'user_id': '351f45edf7084e5c998ac111e45fd359', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 18 02:16:11.828280 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80399) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 18 02:16:11.835714 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-063ff975-4db8-44c5-9d2d-e298d90b8521 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 18 02:16:11.835714 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:16:11.835714 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 18 02:16:11.835714 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 18 02:16:11.835714 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 18 02:16:11.835714 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 18 02:16:11.835714 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 18 02:16:11.835714 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application result = self._query(query) Jul 18 02:16:11.835714 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 18 02:16:11.835714 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application conn.query(q) Jul 18 02:16:11.835714 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 18 02:16:11.835714 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 18 02:16:11.835714 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 18 02:16:11.835714 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application result.read() Jul 18 02:16:11.835714 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 18 02:16:11.840749 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 18 02:16:11.840749 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 18 02:16:11.840749 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 18 02:16:11.840749 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 18 02:16:11.840749 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 18 02:16:11.840749 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 18 02:16:11.840749 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 18 02:16:11.840749 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 18 02:16:11.840749 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:16:11.840749 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 18 02:16:11.840749 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:16:11.840749 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:16:11.840749 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 18 02:16:11.840749 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 18 02:16:11.840749 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 18 02:16:11.840749 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 18 02:16:11.840749 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 18 02:16:11.840749 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application next(self.gen) Jul 18 02:16:11.841750 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 18 02:16:11.841750 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application yield resource Jul 18 02:16:11.841750 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 18 02:16:11.841750 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application next(self.gen) Jul 18 02:16:11.841750 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 18 02:16:11.841750 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self.session.rollback() Jul 18 02:16:11.841750 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 18 02:16:11.841750 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self.force_reraise() Jul 18 02:16:11.841750 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 18 02:16:11.841750 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise self.value Jul 18 02:16:11.841750 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 18 02:16:11.841750 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 18 02:16:11.841750 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 18 02:16:11.841750 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application session.commit() Jul 18 02:16:11.841750 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 18 02:16:11.841750 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 18 02:16:11.841750 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 18 02:16:11.842774 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._prepare_impl() Jul 18 02:16:11.842774 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 18 02:16:11.842774 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self.session.flush() Jul 18 02:16:11.842774 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 18 02:16:11.842774 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._flush(objects) Jul 18 02:16:11.842774 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 18 02:16:11.842774 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 18 02:16:11.842774 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 18 02:16:11.842774 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application compat.raise_( Jul 18 02:16:11.842774 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 18 02:16:11.842774 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception Jul 18 02:16:11.842774 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 18 02:16:11.842774 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application flush_context.execute() Jul 18 02:16:11.842774 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 18 02:16:11.842774 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rec.execute(self) Jul 18 02:16:11.842774 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 18 02:16:11.842774 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 18 02:16:11.843697 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 18 02:16:11.843697 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 18 02:16:11.843697 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 18 02:16:11.843697 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 18 02:16:11.843697 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 18 02:16:11.843697 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 18 02:16:11.843697 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 18 02:16:11.843697 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 18 02:16:11.843697 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 18 02:16:11.843697 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 18 02:16:11.843697 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 18 02:16:11.843697 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 18 02:16:11.843697 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 18 02:16:11.843697 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 18 02:16:11.843697 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 18 02:16:11.843697 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception Jul 18 02:16:11.844573 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 18 02:16:11.844573 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 18 02:16:11.844573 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 18 02:16:11.844573 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 18 02:16:11.844573 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 18 02:16:11.844573 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application result = self._query(query) Jul 18 02:16:11.844573 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 18 02:16:11.844573 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application conn.query(q) Jul 18 02:16:11.844573 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 18 02:16:11.844573 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 18 02:16:11.844573 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 18 02:16:11.844573 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application result.read() Jul 18 02:16:11.844573 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 18 02:16:11.844573 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 18 02:16:11.844573 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 18 02:16:11.844573 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 18 02:16:11.845427 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 18 02:16:11.845427 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 18 02:16:11.845427 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 18 02:16:11.845427 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 18 02:16:11.845427 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 18 02:16:11.845427 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 18 02:16:11.845427 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [parameters: {'user_id': '351f45edf7084e5c998ac111e45fd359', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 18 02:16:11.845427 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 18 02:16:11.845427 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:16:11.845427 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 02:16:11.845427 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:16:11.845427 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:16:11.845427 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:16:11.845427 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:16:11.845427 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:16:11.847254 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:16:11.847254 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:16:11.847254 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:16:11.847254 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:16:11.847254 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:16:11.847254 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:16:11.847254 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:16:11.847254 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 18 02:16:11.847254 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 18 02:16:11.847254 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:16:11.847254 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:16:11.847254 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 02:16:11.847254 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 02:16:11.847254 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 02:16:11.847254 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 02:16:11.847254 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 18 02:16:11.847254 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 18 02:16:11.848461 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 18 02:16:11.848461 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 18 02:16:11.848461 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 18 02:16:11.848461 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 18 02:16:11.848461 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 18 02:16:11.848461 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:16:11.848461 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 483/973] 10.222.0.66 () {60 vars in 1131 bytes} [Mon Jul 18 02:16:11 2022] POST /identity/v3/users => generated 160 bytes in 227 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:16:11.848461 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063ff975-4db8-44c5-9d2d-e298d90b8521 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:11.921572 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:11.934927 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28431b7-4ff3-472e-a905-f3c837b8f7a7 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:11.956102 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063ff975-4db8-44c5-9d2d-e298d90b8521 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:11.980908 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28be76fb-c04f-4c53-b6a8-6dfec0634d8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:11.981808 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28be76fb-c04f-4c53-b6a8-6dfec0634d8b None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:11.982203 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28be76fb-c04f-4c53-b6a8-6dfec0634d8b None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:11.982570 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28be76fb-c04f-4c53-b6a8-6dfec0634d8b None admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:11.983907 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28be76fb-c04f-4c53-b6a8-6dfec0634d8b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:16:11.984225 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28be76fb-c04f-4c53-b6a8-6dfec0634d8b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:11.986116 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28be76fb-c04f-4c53-b6a8-6dfec0634d8b None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:12.000220 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de85c228-477f-4901-83af-8793129d5436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:12.000220 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de85c228-477f-4901-83af-8793129d5436 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:12.000220 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de85c228-477f-4901-83af-8793129d5436 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:12.000220 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de85c228-477f-4901-83af-8793129d5436 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:12.004457 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 491/974] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 18 02:16:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:16:12.005264 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de85c228-477f-4901-83af-8793129d5436 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:12.007174 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de85c228-477f-4901-83af-8793129d5436 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:12.007300 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28be76fb-c04f-4c53-b6a8-6dfec0634d8b None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:12.026664 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-de85c228-477f-4901-83af-8793129d5436 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 18 02:16:12.026664 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:16:12.026664 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:16:12.026664 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:16:12.026664 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:16:12.026664 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:16:12.026664 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:16:12.026664 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:16:12.026664 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:16:12.026664 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:16:12.026664 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:16:12.026664 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:16:12.026664 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 02:16:12.026664 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 02:16:12.026664 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 02:16:12.026664 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 02:16:12.028339 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:16:12.028339 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:16:12.028339 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 02:16:12.028339 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 02:16:12.028339 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 02:16:12.028339 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 02:16:12.028339 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:16:12.028339 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:16:12.028339 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:16:12.028339 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:16:12.028339 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:16:12.028339 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:16:12.028339 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:16:12.028339 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:16:12.028339 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:16:12.028339 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:16:12.028339 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:16:12.028339 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:16:12.030423 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:16:12.030423 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:16:12.030423 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 02:16:12.030423 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 02:16:12.030423 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 02:16:12.030423 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 02:16:12.030423 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 02:16:12.030423 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 02:16:12.030423 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 18 02:16:12.030423 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:16:12.038107 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 484/975] 10.222.0.66 () {56 vars in 1099 bytes} [Mon Jul 18 02:16:11 2022] GET /identity/v3/users/cinder => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:16:12.038107 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de85c228-477f-4901-83af-8793129d5436 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:12.092974 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:12.131875 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28be76fb-c04f-4c53-b6a8-6dfec0634d8b None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:12.150753 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de85c228-477f-4901-83af-8793129d5436 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:12.168910 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:12.173784 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8313b9be-ad49-484d-b5fd-1377f915df69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:12.174785 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8313b9be-ad49-484d-b5fd-1377f915df69 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:12.175209 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8313b9be-ad49-484d-b5fd-1377f915df69 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:12.175646 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8313b9be-ad49-484d-b5fd-1377f915df69 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:12.183790 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8313b9be-ad49-484d-b5fd-1377f915df69 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:12.185453 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8313b9be-ad49-484d-b5fd-1377f915df69 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:12.195309 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94988948-03d5-44e3-aa82-61ad1baf08c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:12.195992 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94988948-03d5-44e3-aa82-61ad1baf08c0 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:12.196799 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94988948-03d5-44e3-aa82-61ad1baf08c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:12.197079 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94988948-03d5-44e3-aa82-61ad1baf08c0 None admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:12.197934 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94988948-03d5-44e3-aa82-61ad1baf08c0 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:16:12.198288 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94988948-03d5-44e3-aa82-61ad1baf08c0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:12.200020 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94988948-03d5-44e3-aa82-61ad1baf08c0 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:12.204529 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-8313b9be-ad49-484d-b5fd-1377f915df69 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 02:16:12.204529 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:16:12.204529 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:16:12.204529 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:16:12.204529 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:16:12.204529 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:16:12.204529 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:16:12.204529 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:16:12.204529 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:16:12.204529 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:16:12.204529 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:16:12.204529 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:16:12.204529 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 02:16:12.204529 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 02:16:12.204529 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 02:16:12.205625 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 02:16:12.205625 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:16:12.205625 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:16:12.205625 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:16:12.205625 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:16:12.205625 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:16:12.205625 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:16:12.205625 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:16:12.205625 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:16:12.205625 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:16:12.205625 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:16:12.205625 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:16:12.205625 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:16:12.205625 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:16:12.205625 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:16:12.205625 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:16:12.205625 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:16:12.206876 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 02:16:12.206876 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 02:16:12.206876 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 02:16:12.206876 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 02:16:12.206876 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 02:16:12.206876 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 02:16:12.206876 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 02:16:12.206876 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:16:12.209121 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 492/976] 10.222.0.66 () {56 vars in 1111 bytes} [Mon Jul 18 02:16:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:16:12.214702 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8313b9be-ad49-484d-b5fd-1377f915df69 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:12.258206 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 485/977] 10.222.0.66 () {56 vars in 1101 bytes} [Mon Jul 18 02:16:12 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 02:16:12.340604 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8313b9be-ad49-484d-b5fd-1377f915df69 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:12.375177 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:12.382079 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76839bc-e7c3-4a1b-9d18-1e98f8fa1d7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:12.383173 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76839bc-e7c3-4a1b-9d18-1e98f8fa1d7b None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:12.383661 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76839bc-e7c3-4a1b-9d18-1e98f8fa1d7b None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:12.384139 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76839bc-e7c3-4a1b-9d18-1e98f8fa1d7b None admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:12.385107 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76839bc-e7c3-4a1b-9d18-1e98f8fa1d7b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:16:12.385636 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76839bc-e7c3-4a1b-9d18-1e98f8fa1d7b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:12.387550 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76839bc-e7c3-4a1b-9d18-1e98f8fa1d7b None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:12.402407 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 493/978] 10.222.0.66 () {56 vars in 1148 bytes} [Mon Jul 18 02:16:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 02:16:12.408770 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94988948-03d5-44e3-aa82-61ad1baf08c0 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:12.518479 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94988948-03d5-44e3-aa82-61ad1baf08c0 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:12.555122 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbe5127-5717-4c6f-9168-82dee6b00002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:12.556928 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbe5127-5717-4c6f-9168-82dee6b00002 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:12.557465 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbe5127-5717-4c6f-9168-82dee6b00002 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:12.557960 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbe5127-5717-4c6f-9168-82dee6b00002 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:12.559028 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbe5127-5717-4c6f-9168-82dee6b00002 None admin] RBAC: Adding query filter params (role.id=a1b1364c0cb94959b01bd1c84faa1b9c, scope.project.id=76f7c0ae36ae49b09dcae4c3e9a875f4, user.id=768f872096584d4093fc616d547aabb6) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:16:12.559566 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbe5127-5717-4c6f-9168-82dee6b00002 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:12.561217 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbe5127-5717-4c6f-9168-82dee6b00002 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:12.567674 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 486/979] 10.222.0.66 () {56 vars in 1374 bytes} [Mon Jul 18 02:16:12 2022] GET /identity/v3/role_assignments?user.id=768f872096584d4093fc616d547aabb6&scope.project.id=76f7c0ae36ae49b09dcae4c3e9a875f4&role.id=a1b1364c0cb94959b01bd1c84faa1b9c => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:16:12.647769 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31a662a-159b-40fe-af55-366999e014c9 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:12.648117 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31a662a-159b-40fe-af55-366999e014c9 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:12.648346 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31a662a-159b-40fe-af55-366999e014c9 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:12.650401 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 494/980] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:16:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:16:12.656481 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2daa772-c1f0-427a-9acd-17e8d6725e3b None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:12.656797 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2daa772-c1f0-427a-9acd-17e8d6725e3b None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:12.657057 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2daa772-c1f0-427a-9acd-17e8d6725e3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:12.675292 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-e2daa772-c1f0-427a-9acd-17e8d6725e3b None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:16:12.676876 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-e2daa772-c1f0-427a-9acd-17e8d6725e3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:16:12.706786 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:12.752426 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 487/981] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:16:12 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 02:16:12.776034 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31a662a-159b-40fe-af55-366999e014c9 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:12.879406 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31a662a-159b-40fe-af55-366999e014c9 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:12.913080 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80478a0-6a83-4f4c-8641-fe9031e857c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:12.913735 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80478a0-6a83-4f4c-8641-fe9031e857c8 None admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:12.914016 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80478a0-6a83-4f4c-8641-fe9031e857c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:12.914316 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80478a0-6a83-4f4c-8641-fe9031e857c8 None admin] PATH_INFO: `/v3/services` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:12.915617 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80478a0-6a83-4f4c-8641-fe9031e857c8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:12.916983 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80478a0-6a83-4f4c-8641-fe9031e857c8 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:12.983554 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 495/982] 10.222.0.66 () {60 vars in 1141 bytes} [Mon Jul 18 02:16:12 2022] POST /identity/v3/services => generated 242 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:16:13.660118 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7fc060-9d40-48ce-bdfb-f0df893671c9 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:13.660669 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7fc060-9d40-48ce-bdfb-f0df893671c9 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:13.660669 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7fc060-9d40-48ce-bdfb-f0df893671c9 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:13.662236 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 488/983] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:16:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:16:13.667798 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb42b52-dd1a-4f19-bcc6-4e54ae8f2a23 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:13.668204 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb42b52-dd1a-4f19-bcc6-4e54ae8f2a23 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:13.668368 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb42b52-dd1a-4f19-bcc6-4e54ae8f2a23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:13.685900 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-efb42b52-dd1a-4f19-bcc6-4e54ae8f2a23 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:16:13.687123 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-efb42b52-dd1a-4f19-bcc6-4e54ae8f2a23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:16:13.738069 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:13.766496 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 496/984] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:16:13 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 02:16:13.788894 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7fc060-9d40-48ce-bdfb-f0df893671c9 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:13.889936 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7fc060-9d40-48ce-bdfb-f0df893671c9 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:13.922897 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc9451c-b503-4fba-ae42-1d47b5ef561c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:13.923693 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc9451c-b503-4fba-ae42-1d47b5ef561c None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:13.923977 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc9451c-b503-4fba-ae42-1d47b5ef561c None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:13.924252 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc9451c-b503-4fba-ae42-1d47b5ef561c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:13.924893 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc9451c-b503-4fba-ae42-1d47b5ef561c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:13.926289 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc9451c-b503-4fba-ae42-1d47b5ef561c None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:13.938733 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-cdc9451c-b503-4fba-ae42-1d47b5ef561c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 02:16:13.938733 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:16:13.938733 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 02:16:13.938733 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 02:16:13.938733 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 02:16:13.938733 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 02:16:13.938733 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 02:16:13.938733 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 02:16:13.938733 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 02:16:13.938733 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:16:13.938733 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 02:16:13.938733 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:16:13.938733 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:16:13.938733 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:16:13.938733 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:16:13.938733 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:16:13.938733 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:16:13.939716 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:16:13.939716 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:16:13.939716 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:16:13.939716 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:16:13.939716 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:16:13.939716 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:16:13.939716 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 02:16:13.939716 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 02:16:13.939716 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 02:16:13.939716 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 02:16:13.939716 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:16:13.939716 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:16:13.939716 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:16:13.939716 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:16:13.939716 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:16:13.939716 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:16:13.940975 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:16:13.940975 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:16:13.940975 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:16:13.940975 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:16:13.940975 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:16:13.940975 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:16:13.940975 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:16:13.940975 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:16:13.940975 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:16:13.940975 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:16:13.940975 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 02:16:13.940975 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 02:16:13.940975 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 02:16:13.940975 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:16:13.942140 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 489/985] 10.222.0.66 () {56 vars in 1126 bytes} [Mon Jul 18 02:16:13 2022] GET /identity/v3/services/object-store => generated 93 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:16:13.945822 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb42b52-dd1a-4f19-bcc6-4e54ae8f2a23 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:14.044379 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb42b52-dd1a-4f19-bcc6-4e54ae8f2a23 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:14.079046 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6805e04-3a78-4d1e-bdfc-6d02f50b84bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:14.079719 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6805e04-3a78-4d1e-bdfc-6d02f50b84bf None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:14.080009 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6805e04-3a78-4d1e-bdfc-6d02f50b84bf None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:14.080285 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6805e04-3a78-4d1e-bdfc-6d02f50b84bf None admin] PATH_INFO: `/v3/services` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:14.080999 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6805e04-3a78-4d1e-bdfc-6d02f50b84bf None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:16:14.081311 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6805e04-3a78-4d1e-bdfc-6d02f50b84bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:14.082740 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6805e04-3a78-4d1e-bdfc-6d02f50b84bf None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:14.088406 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 497/986] 10.222.0.66 () {56 vars in 1122 bytes} [Mon Jul 18 02:16:13 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 02:16:14.094300 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc9451c-b503-4fba-ae42-1d47b5ef561c None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:14.126937 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:14.201599 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc9451c-b503-4fba-ae42-1d47b5ef561c None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:14.236473 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fb03b0-983a-4376-b087-cd11b69d3041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:14.237138 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fb03b0-983a-4376-b087-cd11b69d3041 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:14.237417 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fb03b0-983a-4376-b087-cd11b69d3041 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:14.237688 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fb03b0-983a-4376-b087-cd11b69d3041 None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:14.238413 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fb03b0-983a-4376-b087-cd11b69d3041 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:16:14.238721 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fb03b0-983a-4376-b087-cd11b69d3041 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:14.240091 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fb03b0-983a-4376-b087-cd11b69d3041 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:14.250237 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 490/987] 10.222.0.66 () {56 vars in 1122 bytes} [Mon Jul 18 02:16:14 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:16:14.251816 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6805e04-3a78-4d1e-bdfc-6d02f50b84bf None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:14.296060 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:14.354664 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6805e04-3a78-4d1e-bdfc-6d02f50b84bf None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:14.388699 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f157e879-fde7-4988-89ae-6a6e5f3a0cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:14.389356 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f157e879-fde7-4988-89ae-6a6e5f3a0cba None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:14.389642 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f157e879-fde7-4988-89ae-6a6e5f3a0cba None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:14.389919 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f157e879-fde7-4988-89ae-6a6e5f3a0cba None admin] PATH_INFO: `/v3/endpoints` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:14.390790 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f157e879-fde7-4988-89ae-6a6e5f3a0cba None admin] RBAC: Adding query filter params (interface=admin, service_id=e34e238b3ce0493bad850cc5706a270b, region_id=RegionOne) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:16:14.391107 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f157e879-fde7-4988-89ae-6a6e5f3a0cba None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:14.392482 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f157e879-fde7-4988-89ae-6a6e5f3a0cba None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:14.397924 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 498/988] 10.222.0.66 () {56 vars in 1249 bytes} [Mon Jul 18 02:16:14 2022] GET /identity/v3/endpoints?service_id=e34e238b3ce0493bad850cc5706a270b&interface=admin®ion_id=RegionOne => generated 193 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 02:16:14.403593 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fb03b0-983a-4376-b087-cd11b69d3041 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:14.499515 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:14.510078 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fb03b0-983a-4376-b087-cd11b69d3041 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:14.543408 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72936151-8c13-46b0-a4a9-72a382446b62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:14.544190 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72936151-8c13-46b0-a4a9-72a382446b62 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:14.544486 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72936151-8c13-46b0-a4a9-72a382446b62 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:14.544860 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72936151-8c13-46b0-a4a9-72a382446b62 None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:14.545809 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72936151-8c13-46b0-a4a9-72a382446b62 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:14.547220 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72936151-8c13-46b0-a4a9-72a382446b62 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:14.554523 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 491/989] 10.222.0.66 () {56 vars in 1087 bytes} [Mon Jul 18 02:16:14 2022] GET /identity/v3/services => generated 1727 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:16:14.653932 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da69e9d-a076-4f57-ae19-ced0ede8611b None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:14.654293 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da69e9d-a076-4f57-ae19-ced0ede8611b None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:14.654529 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da69e9d-a076-4f57-ae19-ced0ede8611b None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:14.656071 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 499/990] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:16:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:16:14.661799 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fdb138-2e3f-499b-9556-152494a7742e None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:14.662388 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fdb138-2e3f-499b-9556-152494a7742e None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:14.662388 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fdb138-2e3f-499b-9556-152494a7742e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:14.681093 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-35fdb138-2e3f-499b-9556-152494a7742e None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:16:14.682390 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-35fdb138-2e3f-499b-9556-152494a7742e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:16:14.755263 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 492/991] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:16:14 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 02:16:14.777404 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da69e9d-a076-4f57-ae19-ced0ede8611b None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:14.880022 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da69e9d-a076-4f57-ae19-ced0ede8611b None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:14.882462 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:14.915432 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf612ca-da6e-4552-ba45-d8abc6fb1698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:14.916450 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf612ca-da6e-4552-ba45-d8abc6fb1698 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:14.916733 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf612ca-da6e-4552-ba45-d8abc6fb1698 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:14.917003 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf612ca-da6e-4552-ba45-d8abc6fb1698 None admin] PATH_INFO: `/v3/services/image` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:14.917682 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf612ca-da6e-4552-ba45-d8abc6fb1698 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:14.919171 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf612ca-da6e-4552-ba45-d8abc6fb1698 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:14.931741 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-bdf612ca-da6e-4552-ba45-d8abc6fb1698 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 18 02:16:14.931741 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:16:14.931741 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 02:16:14.931741 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 02:16:14.931741 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 02:16:14.931741 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 02:16:14.931741 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 02:16:14.931741 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 02:16:14.931741 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 18 02:16:14.931741 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:16:14.931741 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 02:16:14.931741 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:16:14.931741 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:16:14.931741 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:16:14.931741 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:16:14.931741 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:16:14.931741 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:16:14.932768 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:16:14.932768 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:16:14.932768 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:16:14.932768 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:16:14.932768 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:16:14.932768 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:16:14.932768 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 02:16:14.932768 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 02:16:14.932768 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 02:16:14.932768 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 02:16:14.932768 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:16:14.932768 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:16:14.932768 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:16:14.932768 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:16:14.932768 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:16:14.932768 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:16:14.933637 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:16:14.933637 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:16:14.933637 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:16:14.933637 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:16:14.933637 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:16:14.933637 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:16:14.933637 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:16:14.933637 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:16:14.933637 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:16:14.933637 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:16:14.933637 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 02:16:14.933637 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 02:16:14.933637 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 18 02:16:14.933637 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:16:14.937820 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 500/992] 10.222.0.66 () {56 vars in 1105 bytes} [Mon Jul 18 02:16:14 2022] GET /identity/v3/services/image => generated 86 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:16:14.939187 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fdb138-2e3f-499b-9556-152494a7742e None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:15.027901 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7dc5a9-a881-418f-9511-36a164c7302c None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:15.028203 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7dc5a9-a881-418f-9511-36a164c7302c None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:15.028407 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7dc5a9-a881-418f-9511-36a164c7302c None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:15.029899 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 501/993] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:16:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 02:16:15.032508 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:15.034687 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2490a46d-939d-4d2c-a72e-b42e60fa9e32 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:15.034982 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2490a46d-939d-4d2c-a72e-b42e60fa9e32 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:15.035260 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2490a46d-939d-4d2c-a72e-b42e60fa9e32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:15.045131 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fdb138-2e3f-499b-9556-152494a7742e None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:15.056005 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-2490a46d-939d-4d2c-a72e-b42e60fa9e32 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:16:15.057117 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-2490a46d-939d-4d2c-a72e-b42e60fa9e32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:16:15.081085 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7b4ce0-8db5-47c4-a9f0-87ace3b32d5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:15.081752 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7b4ce0-8db5-47c4-a9f0-87ace3b32d5a None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:15.082033 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7b4ce0-8db5-47c4-a9f0-87ace3b32d5a None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:15.082339 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7b4ce0-8db5-47c4-a9f0-87ace3b32d5a None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:15.083043 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7b4ce0-8db5-47c4-a9f0-87ace3b32d5a None admin] RBAC: Adding query filter params (name=image) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:16:15.083350 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7b4ce0-8db5-47c4-a9f0-87ace3b32d5a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:15.084727 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7b4ce0-8db5-47c4-a9f0-87ace3b32d5a None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:15.089639 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 493/994] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 18 02:16:14 2022] GET /identity/v3/services?name=image => generated 122 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:16:15.095603 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7b4ce0-8db5-47c4-a9f0-87ace3b32d5a None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:15.137064 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 502/995] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:16:15 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 02:16:15.160182 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2490a46d-939d-4d2c-a72e-b42e60fa9e32 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:15.200970 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7b4ce0-8db5-47c4-a9f0-87ace3b32d5a None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:15.236700 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15b70c6-8afe-4da0-90c7-3c7123b64134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:15.237379 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15b70c6-8afe-4da0-90c7-3c7123b64134 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:15.237665 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15b70c6-8afe-4da0-90c7-3c7123b64134 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:15.237944 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15b70c6-8afe-4da0-90c7-3c7123b64134 None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:15.238703 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15b70c6-8afe-4da0-90c7-3c7123b64134 None admin] RBAC: Adding query filter params (type=image) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:16:15.239014 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15b70c6-8afe-4da0-90c7-3c7123b64134 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:15.240390 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15b70c6-8afe-4da0-90c7-3c7123b64134 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:15.245859 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 494/996] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 18 02:16:15 2022] GET /identity/v3/services?type=image => generated 122 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 02:16:15.263584 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:15.267647 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2490a46d-939d-4d2c-a72e-b42e60fa9e32 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:15.307467 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5000185c-136a-4302-af6f-cba42332e76d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:15.308195 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5000185c-136a-4302-af6f-cba42332e76d None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:15.308479 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5000185c-136a-4302-af6f-cba42332e76d None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:15.308754 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5000185c-136a-4302-af6f-cba42332e76d None admin] PATH_INFO: `/v3/services/compute` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:15.309418 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5000185c-136a-4302-af6f-cba42332e76d None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:15.310883 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5000185c-136a-4302-af6f-cba42332e76d None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:15.324241 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-5000185c-136a-4302-af6f-cba42332e76d None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 18 02:16:15.324241 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:16:15.324241 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 02:16:15.324241 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 02:16:15.324241 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 02:16:15.324241 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 02:16:15.324241 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 02:16:15.324241 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 02:16:15.324241 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 18 02:16:15.324241 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:16:15.324241 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 02:16:15.324241 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:16:15.324241 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:16:15.324241 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:16:15.324241 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:16:15.324241 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:16:15.324241 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:16:15.325299 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:16:15.325299 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:16:15.325299 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:16:15.325299 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:16:15.325299 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:16:15.325299 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:16:15.325299 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 02:16:15.325299 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 02:16:15.325299 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 02:16:15.325299 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 02:16:15.325299 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:16:15.325299 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:16:15.325299 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:16:15.325299 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:16:15.325299 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:16:15.325299 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:16:15.331287 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:16:15.331287 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:16:15.331287 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:16:15.331287 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:16:15.331287 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:16:15.331287 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:16:15.331287 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:16:15.331287 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:16:15.331287 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:16:15.331287 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:16:15.331287 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 02:16:15.331287 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 02:16:15.331287 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 18 02:16:15.331287 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:16:15.331287 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 503/997] 10.222.0.66 () {56 vars in 1111 bytes} [Mon Jul 18 02:16:15 2022] GET /identity/v3/services/compute => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 02:16:15.332092 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15b70c6-8afe-4da0-90c7-3c7123b64134 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:15.348959 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:15.441351 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15b70c6-8afe-4da0-90c7-3c7123b64134 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:15.476178 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdf9a5d-0821-4c33-a286-41d42ff34fb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:15.476845 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdf9a5d-0821-4c33-a286-41d42ff34fb0 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:15.477275 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdf9a5d-0821-4c33-a286-41d42ff34fb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:15.477558 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdf9a5d-0821-4c33-a286-41d42ff34fb0 None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:15.478779 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdf9a5d-0821-4c33-a286-41d42ff34fb0 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:16:15.479091 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdf9a5d-0821-4c33-a286-41d42ff34fb0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:15.480466 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdf9a5d-0821-4c33-a286-41d42ff34fb0 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:15.490006 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 495/998] 10.222.0.66 () {56 vars in 1112 bytes} [Mon Jul 18 02:16:15 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:16:15.491584 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5000185c-136a-4302-af6f-cba42332e76d None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:15.594805 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5000185c-136a-4302-af6f-cba42332e76d None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:15.631034 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c42694a-f33a-47e1-b757-7e3bac984e8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:15.631703 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c42694a-f33a-47e1-b757-7e3bac984e8f None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:15.631982 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c42694a-f33a-47e1-b757-7e3bac984e8f None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:15.632259 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c42694a-f33a-47e1-b757-7e3bac984e8f None admin] PATH_INFO: `/v3/services` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:15.632964 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c42694a-f33a-47e1-b757-7e3bac984e8f None admin] RBAC: Adding query filter params (type=compute) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:16:15.633765 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c42694a-f33a-47e1-b757-7e3bac984e8f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:15.635246 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c42694a-f33a-47e1-b757-7e3bac984e8f None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:15.640512 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 504/999] 10.222.0.66 () {56 vars in 1112 bytes} [Mon Jul 18 02:16:15 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:16:15.646575 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdf9a5d-0821-4c33-a286-41d42ff34fb0 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:15.732163 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:15.755173 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdf9a5d-0821-4c33-a286-41d42ff34fb0 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:15.790643 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2407e9-4cd8-45e6-b263-e6a63401b47e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:15.791303 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2407e9-4cd8-45e6-b263-e6a63401b47e None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:15.791591 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2407e9-4cd8-45e6-b263-e6a63401b47e None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:15.791869 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2407e9-4cd8-45e6-b263-e6a63401b47e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:15.792682 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2407e9-4cd8-45e6-b263-e6a63401b47e None admin] RBAC: Adding query filter params (interface=public, service_id=4b2bbaaa1f7b463b8926bd08283d3470, region_id=RegionOne) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:16:15.793002 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2407e9-4cd8-45e6-b263-e6a63401b47e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:15.794399 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2407e9-4cd8-45e6-b263-e6a63401b47e None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:15.804458 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 496/1000] 10.222.0.66 () {56 vars in 1251 bytes} [Mon Jul 18 02:16:15 2022] GET /identity/v3/endpoints?service_id=4b2bbaaa1f7b463b8926bd08283d3470&interface=public®ion_id=RegionOne => generated 194 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:16:15.805797 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c42694a-f33a-47e1-b757-7e3bac984e8f None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:15.834666 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:15.906956 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c42694a-f33a-47e1-b757-7e3bac984e8f None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:15.940523 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b43ce7-cfea-4174-8fbc-f237c393f010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:15.941193 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b43ce7-cfea-4174-8fbc-f237c393f010 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:15.941479 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b43ce7-cfea-4174-8fbc-f237c393f010 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:15.941750 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b43ce7-cfea-4174-8fbc-f237c393f010 None admin] PATH_INFO: `/v3/services` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:15.942449 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b43ce7-cfea-4174-8fbc-f237c393f010 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:15.943818 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b43ce7-cfea-4174-8fbc-f237c393f010 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:15.958115 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 505/1001] 10.222.0.66 () {56 vars in 1087 bytes} [Mon Jul 18 02:16:15 2022] GET /identity/v3/services => generated 1727 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 02:16:16.583132 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f224277-e5f0-46c2-a21f-13149a4d8e90 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:16.583508 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f224277-e5f0-46c2-a21f-13149a4d8e90 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:16.583709 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f224277-e5f0-46c2-a21f-13149a4d8e90 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:16.585207 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 497/1002] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:16:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:16:16.590240 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5027082-c0e8-4c8c-ab13-908dc738420b None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:16.590532 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5027082-c0e8-4c8c-ab13-908dc738420b None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:16.590808 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5027082-c0e8-4c8c-ab13-908dc738420b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:16.609728 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-a5027082-c0e8-4c8c-ab13-908dc738420b None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:16:16.611305 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-a5027082-c0e8-4c8c-ab13-908dc738420b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:16:16.680363 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 506/1003] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:16:16 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 02:16:16.702912 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f224277-e5f0-46c2-a21f-13149a4d8e90 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:16.804499 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f224277-e5f0-46c2-a21f-13149a4d8e90 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:16.838172 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2568f09f-f533-48f5-8339-f94475e106ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:16.838907 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2568f09f-f533-48f5-8339-f94475e106ba None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:16.839182 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2568f09f-f533-48f5-8339-f94475e106ba None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:16.839452 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2568f09f-f533-48f5-8339-f94475e106ba None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:16.840143 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2568f09f-f533-48f5-8339-f94475e106ba None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:16.841505 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2568f09f-f533-48f5-8339-f94475e106ba None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:16.854125 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-2568f09f-f533-48f5-8339-f94475e106ba None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 02:16:16.854125 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:16:16.854125 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 02:16:16.854125 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 02:16:16.854125 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 02:16:16.854125 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 02:16:16.854125 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 02:16:16.854125 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 02:16:16.854125 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 02:16:16.854125 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:16:16.854125 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 02:16:16.854125 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:16:16.854125 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:16:16.854125 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:16:16.854125 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:16:16.854125 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:16:16.854125 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:16:16.855126 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:16:16.855126 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:16:16.855126 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:16:16.855126 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:16:16.855126 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:16:16.855126 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:16:16.855126 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 02:16:16.855126 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 02:16:16.855126 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 02:16:16.855126 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 02:16:16.855126 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:16:16.855126 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:16:16.855126 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:16:16.855126 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:16:16.855126 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:16:16.855126 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:16:16.855984 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:16:16.855984 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:16:16.855984 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:16:16.855984 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:16:16.855984 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:16:16.855984 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:16:16.855984 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:16:16.855984 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:16:16.855984 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:16:16.855984 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:16:16.855984 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 02:16:16.855984 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 02:16:16.855984 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 02:16:16.855984 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:16:16.858413 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 498/1004] 10.222.0.66 () {56 vars in 1126 bytes} [Mon Jul 18 02:16:16 2022] GET /identity/v3/services/object-store => generated 93 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:16:16.861087 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5027082-c0e8-4c8c-ab13-908dc738420b None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:16.882058 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:16.960216 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5027082-c0e8-4c8c-ab13-908dc738420b None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:16.994650 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcf1b1e-5ece-4d02-a5e1-627edfc263dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:16.995335 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcf1b1e-5ece-4d02-a5e1-627edfc263dd None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:16.995618 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcf1b1e-5ece-4d02-a5e1-627edfc263dd None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:16.995890 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcf1b1e-5ece-4d02-a5e1-627edfc263dd None admin] PATH_INFO: `/v3/services` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:16.996581 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcf1b1e-5ece-4d02-a5e1-627edfc263dd None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:16:16.996884 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcf1b1e-5ece-4d02-a5e1-627edfc263dd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:16.998269 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcf1b1e-5ece-4d02-a5e1-627edfc263dd None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:17.007967 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 507/1005] 10.222.0.66 () {56 vars in 1122 bytes} [Mon Jul 18 02:16:16 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:16:17.009690 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2568f09f-f533-48f5-8339-f94475e106ba None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:17.013546 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:17.116001 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2568f09f-f533-48f5-8339-f94475e106ba None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:17.149887 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8150c8-f685-42c7-b718-71ca0aa99468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:17.150585 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8150c8-f685-42c7-b718-71ca0aa99468 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:17.150878 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8150c8-f685-42c7-b718-71ca0aa99468 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:17.151147 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8150c8-f685-42c7-b718-71ca0aa99468 None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:17.151849 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8150c8-f685-42c7-b718-71ca0aa99468 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:16:17.152160 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8150c8-f685-42c7-b718-71ca0aa99468 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:17.153530 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8150c8-f685-42c7-b718-71ca0aa99468 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:17.158645 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 499/1006] 10.222.0.66 () {56 vars in 1122 bytes} [Mon Jul 18 02:16:17 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 02:16:17.164052 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcf1b1e-5ece-4d02-a5e1-627edfc263dd None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:17.255899 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:17.272311 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcf1b1e-5ece-4d02-a5e1-627edfc263dd None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:17.275264 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e8b8ed-d6e7-42f7-863a-c71991527637 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:17.275558 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e8b8ed-d6e7-42f7-863a-c71991527637 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:17.275841 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e8b8ed-d6e7-42f7-863a-c71991527637 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:17.277251 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 500/1007] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:16:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 02:16:17.282477 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef193bb-5903-45fc-9f6f-73459476ff6c None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:17.282766 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef193bb-5903-45fc-9f6f-73459476ff6c None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:17.283046 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef193bb-5903-45fc-9f6f-73459476ff6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:17.300438 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-cef193bb-5903-45fc-9f6f-73459476ff6c None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:16:17.301722 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-cef193bb-5903-45fc-9f6f-73459476ff6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:16:17.312296 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82504c68-fb75-4ce9-b02f-c7157c0cceb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:17.312952 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82504c68-fb75-4ce9-b02f-c7157c0cceb3 None admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:17.313241 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82504c68-fb75-4ce9-b02f-c7157c0cceb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:17.313519 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82504c68-fb75-4ce9-b02f-c7157c0cceb3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:17.314286 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82504c68-fb75-4ce9-b02f-c7157c0cceb3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:17.315643 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82504c68-fb75-4ce9-b02f-c7157c0cceb3 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:17.363270 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:17.369363 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 501/1008] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:16:17 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 02:16:17.390117 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 508/1009] 10.222.0.66 () {60 vars in 1144 bytes} [Mon Jul 18 02:16:17 2022] POST /identity/v3/endpoints => generated 325 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:16:17.391904 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef193bb-5903-45fc-9f6f-73459476ff6c None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:17.507347 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef193bb-5903-45fc-9f6f-73459476ff6c None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:17.556576 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508bc69a-873d-4020-a244-9c7430e90085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:17.557297 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508bc69a-873d-4020-a244-9c7430e90085 None admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:17.557587 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508bc69a-873d-4020-a244-9c7430e90085 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:17.557865 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508bc69a-873d-4020-a244-9c7430e90085 None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:17.558700 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508bc69a-873d-4020-a244-9c7430e90085 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:17.560075 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508bc69a-873d-4020-a244-9c7430e90085 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:17.663945 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 502/1010] 10.222.0.66 () {60 vars in 1141 bytes} [Mon Jul 18 02:16:17 2022] POST /identity/v3/services => generated 242 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:16:17.990056 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7f307e-7c6a-4987-990e-8af5d088ec25 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:17.990579 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7f307e-7c6a-4987-990e-8af5d088ec25 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:17.990579 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7f307e-7c6a-4987-990e-8af5d088ec25 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:17.991972 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 509/1011] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:16:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:16:17.997238 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e187e1-139d-408a-8959-4b4e18610d85 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:17.997654 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e187e1-139d-408a-8959-4b4e18610d85 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:17.997816 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e187e1-139d-408a-8959-4b4e18610d85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:18.015959 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-36e187e1-139d-408a-8959-4b4e18610d85 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:16:18.017129 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-36e187e1-139d-408a-8959-4b4e18610d85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:16:18.067642 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:18.092344 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 503/1012] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:16:17 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 02:16:18.114687 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7f307e-7c6a-4987-990e-8af5d088ec25 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:18.221638 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7f307e-7c6a-4987-990e-8af5d088ec25 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:18.255337 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50706ed-ca0c-4c30-a9d0-fdc63267696e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:18.256049 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50706ed-ca0c-4c30-a9d0-fdc63267696e None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:18.256334 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50706ed-ca0c-4c30-a9d0-fdc63267696e None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:18.256609 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50706ed-ca0c-4c30-a9d0-fdc63267696e None admin] PATH_INFO: `/v3/services/compute` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:18.257255 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50706ed-ca0c-4c30-a9d0-fdc63267696e None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:18.259250 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50706ed-ca0c-4c30-a9d0-fdc63267696e None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:18.270763 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-d50706ed-ca0c-4c30-a9d0-fdc63267696e None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 18 02:16:18.270763 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:16:18.270763 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 02:16:18.270763 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 02:16:18.270763 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 02:16:18.270763 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 02:16:18.270763 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 02:16:18.270763 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 02:16:18.270763 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 18 02:16:18.270763 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:16:18.270763 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 02:16:18.270763 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:16:18.270763 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:16:18.270763 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:16:18.270763 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:16:18.270763 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:16:18.270763 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:16:18.271793 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:16:18.271793 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:16:18.271793 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:16:18.271793 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:16:18.271793 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:16:18.271793 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:16:18.271793 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 02:16:18.271793 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 02:16:18.271793 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 02:16:18.271793 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 02:16:18.271793 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:16:18.271793 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:16:18.271793 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:16:18.271793 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:16:18.271793 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:16:18.271793 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:16:18.272662 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:16:18.272662 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:16:18.272662 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:16:18.272662 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:16:18.272662 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:16:18.272662 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:16:18.272662 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:16:18.272662 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:16:18.272662 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:16:18.272662 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:16:18.272662 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 02:16:18.272662 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 02:16:18.272662 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 18 02:16:18.272662 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:16:18.272662 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 510/1013] 10.222.0.66 () {56 vars in 1111 bytes} [Mon Jul 18 02:16:18 2022] GET /identity/v3/services/compute => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:16:18.279933 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e187e1-139d-408a-8959-4b4e18610d85 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:18.382811 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e187e1-139d-408a-8959-4b4e18610d85 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:18.419268 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e459d6-7169-476e-860d-09b06841ab72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:18.420192 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e459d6-7169-476e-860d-09b06841ab72 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:18.420475 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e459d6-7169-476e-860d-09b06841ab72 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:18.420752 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e459d6-7169-476e-860d-09b06841ab72 None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:18.421451 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e459d6-7169-476e-860d-09b06841ab72 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:16:18.421755 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e459d6-7169-476e-860d-09b06841ab72 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:18.423156 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e459d6-7169-476e-860d-09b06841ab72 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:18.432950 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 504/1014] 10.222.0.66 () {56 vars in 1112 bytes} [Mon Jul 18 02:16:18 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:16:18.434317 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50706ed-ca0c-4c30-a9d0-fdc63267696e None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:18.446605 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:18.545207 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50706ed-ca0c-4c30-a9d0-fdc63267696e None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:18.579680 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98741ac8-8380-4386-88fa-513dcee2175c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:18.580344 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98741ac8-8380-4386-88fa-513dcee2175c None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:18.580631 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98741ac8-8380-4386-88fa-513dcee2175c None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:18.580911 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98741ac8-8380-4386-88fa-513dcee2175c None admin] PATH_INFO: `/v3/services` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:18.581618 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98741ac8-8380-4386-88fa-513dcee2175c None admin] RBAC: Adding query filter params (type=compute) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:16:18.581930 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98741ac8-8380-4386-88fa-513dcee2175c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:18.583346 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98741ac8-8380-4386-88fa-513dcee2175c None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:18.593502 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 511/1015] 10.222.0.66 () {56 vars in 1112 bytes} [Mon Jul 18 02:16:18 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:16:18.594691 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e459d6-7169-476e-860d-09b06841ab72 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:18.643837 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:18.699576 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e459d6-7169-476e-860d-09b06841ab72 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:18.733286 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28aaae0-99e7-4123-8c6f-2e52314c0c70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:18.733949 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28aaae0-99e7-4123-8c6f-2e52314c0c70 None admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:18.734277 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28aaae0-99e7-4123-8c6f-2e52314c0c70 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:18.734559 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28aaae0-99e7-4123-8c6f-2e52314c0c70 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:18.735296 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28aaae0-99e7-4123-8c6f-2e52314c0c70 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:18.736668 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28aaae0-99e7-4123-8c6f-2e52314c0c70 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:18.812704 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 505/1016] 10.222.0.66 () {60 vars in 1144 bytes} [Mon Jul 18 02:16:18 2022] POST /identity/v3/endpoints => generated 334 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:16:19.459065 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9f921b-0c52-4be7-a1e8-c5b4a6050176 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:19.459644 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9f921b-0c52-4be7-a1e8-c5b4a6050176 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:19.459644 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9f921b-0c52-4be7-a1e8-c5b4a6050176 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:19.461172 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 512/1017] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:16:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:16:19.466893 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc398bc-ab59-4fcd-b672-aaf9feedc2e0 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:19.467355 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc398bc-ab59-4fcd-b672-aaf9feedc2e0 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:19.467451 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc398bc-ab59-4fcd-b672-aaf9feedc2e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:19.484653 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-4cc398bc-ab59-4fcd-b672-aaf9feedc2e0 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:16:19.485838 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-4cc398bc-ab59-4fcd-b672-aaf9feedc2e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:16:19.565424 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 506/1018] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:16:19 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 02:16:19.589149 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9f921b-0c52-4be7-a1e8-c5b4a6050176 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:19.670643 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:19.698431 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814a8cbb-b20b-4eb2-a94e-7f4701cb6618 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:19.698716 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814a8cbb-b20b-4eb2-a94e-7f4701cb6618 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:19.699004 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814a8cbb-b20b-4eb2-a94e-7f4701cb6618 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:19.700514 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 507/1019] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:16:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:16:19.703249 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9f921b-0c52-4be7-a1e8-c5b4a6050176 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:19.706004 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3326d8ba-4b3e-4128-98e7-095080d13b0a None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:19.706324 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3326d8ba-4b3e-4128-98e7-095080d13b0a None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:19.706607 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3326d8ba-4b3e-4128-98e7-095080d13b0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:19.724220 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-3326d8ba-4b3e-4128-98e7-095080d13b0a None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:16:19.725603 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-3326d8ba-4b3e-4128-98e7-095080d13b0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:16:19.736767 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41882f2-b47a-4cfe-8453-ad130536b88f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:19.737552 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41882f2-b47a-4cfe-8453-ad130536b88f None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:19.737833 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41882f2-b47a-4cfe-8453-ad130536b88f None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:19.738127 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41882f2-b47a-4cfe-8453-ad130536b88f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:19.739284 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41882f2-b47a-4cfe-8453-ad130536b88f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:19.740705 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41882f2-b47a-4cfe-8453-ad130536b88f None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:19.747662 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 513/1020] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 18 02:16:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:16:19.748558 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41882f2-b47a-4cfe-8453-ad130536b88f None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:19.749486 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:19.800598 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 508/1021] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:16:19 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 02:16:19.823322 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3326d8ba-4b3e-4128-98e7-095080d13b0a None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:19.857475 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41882f2-b47a-4cfe-8453-ad130536b88f None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:19.893704 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7833f3b6-be7d-4d93-af1d-ba349a5c1307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:19.894391 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7833f3b6-be7d-4d93-af1d-ba349a5c1307 None admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:19.894681 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7833f3b6-be7d-4d93-af1d-ba349a5c1307 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:19.894963 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7833f3b6-be7d-4d93-af1d-ba349a5c1307 None admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:19.895672 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7833f3b6-be7d-4d93-af1d-ba349a5c1307 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:19.897268 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7833f3b6-be7d-4d93-af1d-ba349a5c1307 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:19.907966 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:19.926507 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3326d8ba-4b3e-4128-98e7-095080d13b0a None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:19.960999 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583d3092-333b-4000-8d94-2fa35e7562ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:19.961978 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583d3092-333b-4000-8d94-2fa35e7562ee None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:19.962290 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583d3092-333b-4000-8d94-2fa35e7562ee None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:19.962567 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583d3092-333b-4000-8d94-2fa35e7562ee None admin] PATH_INFO: `/v3/services/image` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:19.963213 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583d3092-333b-4000-8d94-2fa35e7562ee None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:19.964588 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583d3092-333b-4000-8d94-2fa35e7562ee None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:19.976080 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-583d3092-333b-4000-8d94-2fa35e7562ee None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 18 02:16:19.976080 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:16:19.976080 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 02:16:19.976080 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 02:16:19.976080 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 02:16:19.976080 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 02:16:19.976080 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 02:16:19.976080 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 02:16:19.976080 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 18 02:16:19.976080 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:16:19.976080 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 02:16:19.976080 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:16:19.976080 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:16:19.976080 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:16:19.976080 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:16:19.976080 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:16:19.976080 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:16:19.977436 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:16:19.977436 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:16:19.977436 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:16:19.977436 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:16:19.977436 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:16:19.977436 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:16:19.977436 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 02:16:19.977436 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 02:16:19.977436 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 02:16:19.977436 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 02:16:19.977436 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:16:19.977436 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:16:19.977436 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:16:19.977436 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:16:19.977436 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:16:19.977436 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:16:19.978377 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:16:19.978377 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:16:19.978377 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:16:19.978377 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:16:19.978377 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:16:19.978377 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:16:19.978377 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:16:19.978377 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:16:19.978377 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:16:19.978377 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:16:19.978377 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 02:16:19.978377 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 02:16:19.978377 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 18 02:16:19.978377 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:16:19.978377 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 509/1022] 10.222.0.66 () {56 vars in 1105 bytes} [Mon Jul 18 02:16:19 2022] GET /identity/v3/services/image => generated 86 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 02:16:19.983077 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583d3092-333b-4000-8d94-2fa35e7562ee None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:19.995684 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 514/1023] 10.222.0.66 () {60 vars in 1141 bytes} [Mon Jul 18 02:16:19 2022] POST /identity/v3/projects => generated 311 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:16:20.009549 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:20.097068 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583d3092-333b-4000-8d94-2fa35e7562ee None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:20.134950 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fd9d5e-e908-4362-a812-8616e1f7d7b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:20.135639 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fd9d5e-e908-4362-a812-8616e1f7d7b7 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:20.135924 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fd9d5e-e908-4362-a812-8616e1f7d7b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:20.136203 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fd9d5e-e908-4362-a812-8616e1f7d7b7 None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:20.136919 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fd9d5e-e908-4362-a812-8616e1f7d7b7 None admin] RBAC: Adding query filter params (name=image) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:16:20.137227 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fd9d5e-e908-4362-a812-8616e1f7d7b7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:20.138663 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fd9d5e-e908-4362-a812-8616e1f7d7b7 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:20.144416 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 510/1024] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 18 02:16:19 2022] GET /identity/v3/services?name=image => generated 122 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 02:16:20.150103 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7833f3b6-be7d-4d93-af1d-ba349a5c1307 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:20.256973 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7833f3b6-be7d-4d93-af1d-ba349a5c1307 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:20.291694 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ad2c79-83a2-4495-a9e9-48a3cc430ce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:20.292359 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ad2c79-83a2-4495-a9e9-48a3cc430ce1 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:20.292645 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ad2c79-83a2-4495-a9e9-48a3cc430ce1 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:20.292924 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ad2c79-83a2-4495-a9e9-48a3cc430ce1 None admin] PATH_INFO: `/v3/services` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:20.293647 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ad2c79-83a2-4495-a9e9-48a3cc430ce1 None admin] RBAC: Adding query filter params (type=image) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:16:20.293939 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ad2c79-83a2-4495-a9e9-48a3cc430ce1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:20.295616 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ad2c79-83a2-4495-a9e9-48a3cc430ce1 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:20.300730 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 515/1025] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 18 02:16:20 2022] GET /identity/v3/services?type=image => generated 350 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:16:20.307090 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fd9d5e-e908-4362-a812-8616e1f7d7b7 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:20.403334 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:20.419976 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fd9d5e-e908-4362-a812-8616e1f7d7b7 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:20.455003 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc2b777-1e05-4f33-b48c-e8bb6d264ebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:20.455679 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc2b777-1e05-4f33-b48c-e8bb6d264ebd None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:20.455965 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc2b777-1e05-4f33-b48c-e8bb6d264ebd None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:20.456243 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc2b777-1e05-4f33-b48c-e8bb6d264ebd None admin] PATH_INFO: `/v3/endpoints` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:20.457036 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc2b777-1e05-4f33-b48c-e8bb6d264ebd None admin] RBAC: Adding query filter params (interface=public, service_id=e03682540b1c4ccc97586c36dadc7d35, region_id=RegionOne) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:16:20.457352 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc2b777-1e05-4f33-b48c-e8bb6d264ebd None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:20.458759 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc2b777-1e05-4f33-b48c-e8bb6d264ebd None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:20.469386 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 511/1026] 10.222.0.66 () {56 vars in 1251 bytes} [Mon Jul 18 02:16:20 2022] GET /identity/v3/endpoints?service_id=e03682540b1c4ccc97586c36dadc7d35&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:16:20.471139 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ad2c79-83a2-4495-a9e9-48a3cc430ce1 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:20.532478 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:20.578040 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ad2c79-83a2-4495-a9e9-48a3cc430ce1 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:20.621191 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3838ed-06bf-4c6e-8811-1d82fcd553f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:20.622119 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3838ed-06bf-4c6e-8811-1d82fcd553f9 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:20.622607 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3838ed-06bf-4c6e-8811-1d82fcd553f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:20.623034 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3838ed-06bf-4c6e-8811-1d82fcd553f9 None admin] PATH_INFO: `/v3/services` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:20.623882 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3838ed-06bf-4c6e-8811-1d82fcd553f9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:20.625397 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3838ed-06bf-4c6e-8811-1d82fcd553f9 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:20.638483 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 516/1027] 10.222.0.66 () {56 vars in 1087 bytes} [Mon Jul 18 02:16:20 2022] GET /identity/v3/services => generated 1957 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:16:20.937683 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735a746f-bfe1-4c34-96f9-112836956666 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:20.937976 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735a746f-bfe1-4c34-96f9-112836956666 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:20.938293 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735a746f-bfe1-4c34-96f9-112836956666 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:20.939684 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 512/1028] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:16:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:16:20.944898 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa13dbb-d0df-4c9b-b6d3-ea4d0c676014 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:20.945224 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa13dbb-d0df-4c9b-b6d3-ea4d0c676014 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:20.945608 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa13dbb-d0df-4c9b-b6d3-ea4d0c676014 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:20.961860 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-9aa13dbb-d0df-4c9b-b6d3-ea4d0c676014 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:16:20.963071 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-9aa13dbb-d0df-4c9b-b6d3-ea4d0c676014 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:16:21.029147 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 517/1029] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:16:20 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 02:16:21.052968 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735a746f-bfe1-4c34-96f9-112836956666 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:21.160166 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735a746f-bfe1-4c34-96f9-112836956666 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:21.196062 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b3d8bf-848d-4ca2-a2c5-6ed9e81b7a0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:21.196791 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b3d8bf-848d-4ca2-a2c5-6ed9e81b7a0b None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:21.197071 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b3d8bf-848d-4ca2-a2c5-6ed9e81b7a0b None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:21.197343 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b3d8bf-848d-4ca2-a2c5-6ed9e81b7a0b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:21.207587 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b3d8bf-848d-4ca2-a2c5-6ed9e81b7a0b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 02:16:21.215876 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-f3b3d8bf-848d-4ca2-a2c5-6ed9e81b7a0b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 02:16:21.215876 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:16:21.215876 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:16:21.215876 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:16:21.215876 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:16:21.215876 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:16:21.215876 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:16:21.215876 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:16:21.215876 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:16:21.215876 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:16:21.215876 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:16:21.215876 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:16:21.215876 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 02:16:21.215876 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 02:16:21.215876 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 02:16:21.215876 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 02:16:21.217319 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 02:16:21.217319 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 02:16:21.217319 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 02:16:21.217319 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 02:16:21.217319 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:16:21.217319 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:16:21.217319 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:16:21.217319 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:16:21.217319 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:16:21.217319 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:16:21.217319 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:16:21.217319 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:16:21.217319 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:16:21.217319 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:16:21.217319 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:16:21.217319 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:16:21.218256 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:16:21.218256 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:16:21.218256 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:16:21.218256 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:16:21.218256 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 02:16:21.218256 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 02:16:21.218256 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 02:16:21.218256 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 02:16:21.218256 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 02:16:21.218256 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 02:16:21.218256 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 02:16:21.218256 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:16:21.221961 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 513/1030] 10.222.0.66 () {56 vars in 1120 bytes} [Mon Jul 18 02:16:21 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:16:21.223241 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa13dbb-d0df-4c9b-b6d3-ea4d0c676014 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:21.275745 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:21.329009 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa13dbb-d0df-4c9b-b6d3-ea4d0c676014 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:21.362915 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33ac267-ce2d-4945-b197-55940a3109ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:21.363592 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33ac267-ce2d-4945-b197-55940a3109ec None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:21.363879 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33ac267-ce2d-4945-b197-55940a3109ec None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:21.364159 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33ac267-ce2d-4945-b197-55940a3109ec None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:21.364900 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33ac267-ce2d-4945-b197-55940a3109ec None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:16:21.365209 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33ac267-ce2d-4945-b197-55940a3109ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:21.366610 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33ac267-ce2d-4945-b197-55940a3109ec None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:21.380144 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 518/1031] 10.222.0.66 () {56 vars in 1115 bytes} [Mon Jul 18 02:16:21 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:16:21.381708 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b3d8bf-848d-4ca2-a2c5-6ed9e81b7a0b None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:21.393791 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:21.483579 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b3d8bf-848d-4ca2-a2c5-6ed9e81b7a0b None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:21.517874 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd42f74-c885-4eb6-bdd6-c3a52f97e9f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:21.518696 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd42f74-c885-4eb6-bdd6-c3a52f97e9f9 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:21.518983 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd42f74-c885-4eb6-bdd6-c3a52f97e9f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:21.519264 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd42f74-c885-4eb6-bdd6-c3a52f97e9f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:21.529803 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd42f74-c885-4eb6-bdd6-c3a52f97e9f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:21.531308 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd42f74-c885-4eb6-bdd6-c3a52f97e9f9 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:21.548528 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-ffd42f74-c885-4eb6-bdd6-c3a52f97e9f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:16:21.548528 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:16:21.548528 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:16:21.548528 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:16:21.548528 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:16:21.548528 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:16:21.548528 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:16:21.548528 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:16:21.548528 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:16:21.548528 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:16:21.548528 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:16:21.548528 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:16:21.548528 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 02:16:21.548528 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 02:16:21.548528 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 02:16:21.549462 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 02:16:21.549462 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:16:21.549462 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:16:21.549462 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:16:21.549462 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:16:21.549462 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:16:21.549462 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:16:21.549462 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:16:21.549462 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:16:21.549462 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:16:21.549462 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:16:21.549462 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:16:21.549462 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:16:21.549462 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:16:21.549462 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:16:21.549462 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:16:21.549462 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:16:21.555939 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 02:16:21.555939 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 02:16:21.555939 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:16:21.555939 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:16:21.555939 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 514/1032] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 18 02:16:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 02:16:21.557721 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33ac267-ce2d-4945-b197-55940a3109ec None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:21.665387 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33ac267-ce2d-4945-b197-55940a3109ec None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:21.678938 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:21.700224 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726fa634-0791-477d-b305-907df214833a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:21.700889 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726fa634-0791-477d-b305-907df214833a None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:21.701173 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726fa634-0791-477d-b305-907df214833a None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:21.701506 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726fa634-0791-477d-b305-907df214833a None admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:21.702257 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726fa634-0791-477d-b305-907df214833a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:16:21.702567 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726fa634-0791-477d-b305-907df214833a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:21.703941 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726fa634-0791-477d-b305-907df214833a None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:21.718132 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 519/1033] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 18 02:16:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:16:21.722029 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd42f74-c885-4eb6-bdd6-c3a52f97e9f9 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:21.784567 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:21.827761 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd42f74-c885-4eb6-bdd6-c3a52f97e9f9 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:21.861422 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3fc628-31ff-4e08-9dba-11c6941a4863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:21.862404 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3fc628-31ff-4e08-9dba-11c6941a4863 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:21.862692 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3fc628-31ff-4e08-9dba-11c6941a4863 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:21.862969 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3fc628-31ff-4e08-9dba-11c6941a4863 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:21.871973 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3fc628-31ff-4e08-9dba-11c6941a4863 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:21.873424 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3fc628-31ff-4e08-9dba-11c6941a4863 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:21.887274 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-ca3fc628-31ff-4e08-9dba-11c6941a4863 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 18 02:16:21.887274 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:16:21.887274 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:16:21.887274 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:16:21.887274 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:16:21.887274 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:16:21.887274 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:16:21.887274 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:16:21.887274 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:16:21.887274 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:16:21.887274 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:16:21.887274 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:16:21.887274 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 02:16:21.887274 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 02:16:21.887274 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 02:16:21.887274 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 02:16:21.888225 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:16:21.888225 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:16:21.888225 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 02:16:21.888225 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 02:16:21.888225 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 02:16:21.888225 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 02:16:21.888225 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:16:21.888225 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:16:21.888225 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:16:21.888225 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:16:21.888225 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:16:21.888225 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:16:21.888225 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:16:21.888225 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:16:21.888225 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:16:21.888225 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:16:21.888225 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:16:21.888225 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:16:21.889206 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:16:21.889206 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:16:21.889206 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 02:16:21.889206 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 02:16:21.889206 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 02:16:21.889206 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 02:16:21.889206 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 02:16:21.889206 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 02:16:21.889206 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 18 02:16:21.889206 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:16:21.893159 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 515/1034] 10.222.0.66 () {56 vars in 1093 bytes} [Mon Jul 18 02:16:21 2022] GET /identity/v3/users/nova => generated 82 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:16:21.894624 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726fa634-0791-477d-b305-907df214833a None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:21.994797 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726fa634-0791-477d-b305-907df214833a None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:22.028310 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d839c9a4-3d93-43d5-990d-e9cd2f3fa716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:22.028981 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d839c9a4-3d93-43d5-990d-e9cd2f3fa716 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:22.029261 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d839c9a4-3d93-43d5-990d-e9cd2f3fa716 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:22.029535 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d839c9a4-3d93-43d5-990d-e9cd2f3fa716 None admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:22.030397 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d839c9a4-3d93-43d5-990d-e9cd2f3fa716 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:16:22.030759 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d839c9a4-3d93-43d5-990d-e9cd2f3fa716 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:22.032190 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d839c9a4-3d93-43d5-990d-e9cd2f3fa716 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:22.054922 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 520/1035] 10.222.0.66 () {56 vars in 1133 bytes} [Mon Jul 18 02:16:21 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 02:16:22.056195 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3fc628-31ff-4e08-9dba-11c6941a4863 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:22.080792 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28919e6-6f80-460f-a448-9dc6a0007011 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:22.081094 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28919e6-6f80-460f-a448-9dc6a0007011 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:22.081360 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28919e6-6f80-460f-a448-9dc6a0007011 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:22.082911 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 521/1036] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:16:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 02:16:22.087717 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7c0d30-393f-4d98-ab1e-c9665ba16784 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:22.088001 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7c0d30-393f-4d98-ab1e-c9665ba16784 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:22.088273 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7c0d30-393f-4d98-ab1e-c9665ba16784 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:22.104816 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-4f7c0d30-393f-4d98-ab1e-c9665ba16784 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:16:22.105990 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-4f7c0d30-393f-4d98-ab1e-c9665ba16784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:16:22.131156 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:22.169021 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3fc628-31ff-4e08-9dba-11c6941a4863 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:22.176979 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 522/1037] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:16:22 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 02:16:22.192016 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:22.201668 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7c0d30-393f-4d98-ab1e-c9665ba16784 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:22.206254 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8f8eb7-9010-4495-9f25-34c4061aa7eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:22.206910 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8f8eb7-9010-4495-9f25-34c4061aa7eb None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:22.207213 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8f8eb7-9010-4495-9f25-34c4061aa7eb None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:22.207466 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8f8eb7-9010-4495-9f25-34c4061aa7eb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:22.282915 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8f8eb7-9010-4495-9f25-34c4061aa7eb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:22.284366 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8f8eb7-9010-4495-9f25-34c4061aa7eb None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:22.308308 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7c0d30-393f-4d98-ab1e-c9665ba16784 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:22.343307 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ae0b6c-bddc-4bee-b52f-ec31b79d95fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:22.344099 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ae0b6c-bddc-4bee-b52f-ec31b79d95fb None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:22.344401 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ae0b6c-bddc-4bee-b52f-ec31b79d95fb None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:22.344685 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ae0b6c-bddc-4bee-b52f-ec31b79d95fb None admin] PATH_INFO: `/v3/domains/default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:22.350500 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ae0b6c-bddc-4bee-b52f-ec31b79d95fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:22.350500 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ae0b6c-bddc-4bee-b52f-ec31b79d95fb None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:22.350500 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 523/1038] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 18 02:16:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:16:22.355410 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ae0b6c-bddc-4bee-b52f-ec31b79d95fb None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:22.383544 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:22.394823 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 516/1039] 10.222.0.66 () {58 vars in 1282 bytes} [Mon Jul 18 02:16:22 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 02:16:22.396133 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8f8eb7-9010-4495-9f25-34c4061aa7eb None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:22.421261 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:22.461172 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ae0b6c-bddc-4bee-b52f-ec31b79d95fb None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:22.496871 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff22dac-979c-45d8-aca9-e3e524b13192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:22.497538 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff22dac-979c-45d8-aca9-e3e524b13192 None admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:22.497818 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff22dac-979c-45d8-aca9-e3e524b13192 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:22.498113 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff22dac-979c-45d8-aca9-e3e524b13192 None admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:22.498815 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff22dac-979c-45d8-aca9-e3e524b13192 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:22.500247 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff22dac-979c-45d8-aca9-e3e524b13192 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:22.514160 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8f8eb7-9010-4495-9f25-34c4061aa7eb None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:22.548853 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a5beaa-ea61-4929-9294-cd6bc6cc9b2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:22.549659 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a5beaa-ea61-4929-9294-cd6bc6cc9b2a None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:22.549935 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a5beaa-ea61-4929-9294-cd6bc6cc9b2a None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:22.550252 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a5beaa-ea61-4929-9294-cd6bc6cc9b2a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:22.560314 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a5beaa-ea61-4929-9294-cd6bc6cc9b2a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:22.561697 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a5beaa-ea61-4929-9294-cd6bc6cc9b2a None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:22.579458 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-a3a5beaa-ea61-4929-9294-cd6bc6cc9b2a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:16:22.579458 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:16:22.579458 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:16:22.579458 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:16:22.579458 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:16:22.579458 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:16:22.579458 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:16:22.579458 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:16:22.579458 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:16:22.579458 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:16:22.579458 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:16:22.579458 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:16:22.579458 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 02:16:22.579458 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 02:16:22.579458 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 02:16:22.586477 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 02:16:22.586477 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:16:22.586477 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:16:22.586477 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:16:22.586477 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:16:22.586477 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:16:22.586477 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:16:22.586477 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:16:22.586477 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:16:22.586477 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:16:22.586477 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:16:22.586477 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:16:22.586477 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:16:22.586477 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:16:22.586477 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:16:22.586477 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:16:22.586477 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:16:22.587539 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 02:16:22.587539 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 02:16:22.587539 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:16:22.587539 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:16:22.587539 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 517/1040] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 18 02:16:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 02:16:22.587539 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a5beaa-ea61-4929-9294-cd6bc6cc9b2a None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:22.626118 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 524/1041] 10.222.0.66 () {60 vars in 1132 bytes} [Mon Jul 18 02:16:22 2022] POST /identity/v3/users => generated 285 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 02:16:22.665977 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:22.681828 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae2f459-371b-45b3-a40a-a032111efc6a None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:22.682161 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae2f459-371b-45b3-a40a-a032111efc6a None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:22.682445 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae2f459-371b-45b3-a40a-a032111efc6a None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:22.683916 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 525/1042] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:16:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:16:22.689665 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385a1112-c201-43d9-ab2e-faf5b6199200 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:22.689971 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385a1112-c201-43d9-ab2e-faf5b6199200 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:22.690299 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385a1112-c201-43d9-ab2e-faf5b6199200 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:22.709220 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-385a1112-c201-43d9-ab2e-faf5b6199200 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:16:22.710507 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-385a1112-c201-43d9-ab2e-faf5b6199200 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:16:22.713098 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a5beaa-ea61-4929-9294-cd6bc6cc9b2a None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:22.751934 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f85837b-f877-47c1-8202-5b6eef016c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:22.753095 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f85837b-f877-47c1-8202-5b6eef016c82 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:22.753095 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f85837b-f877-47c1-8202-5b6eef016c82 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:22.753305 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f85837b-f877-47c1-8202-5b6eef016c82 None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:22.754004 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f85837b-f877-47c1-8202-5b6eef016c82 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:16:22.754341 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f85837b-f877-47c1-8202-5b6eef016c82 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:22.755749 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f85837b-f877-47c1-8202-5b6eef016c82 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:22.770704 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 518/1043] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 18 02:16:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:16:22.776616 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f85837b-f877-47c1-8202-5b6eef016c82 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:22.789399 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 526/1044] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:16:22 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 02:16:22.813287 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385a1112-c201-43d9-ab2e-faf5b6199200 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:22.828976 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:22.888062 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f85837b-f877-47c1-8202-5b6eef016c82 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:22.922933 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:22.926248 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385a1112-c201-43d9-ab2e-faf5b6199200 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:22.927983 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9733d42a-0035-412c-a660-54db0fb4480e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:22.928779 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9733d42a-0035-412c-a660-54db0fb4480e None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:22.929064 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9733d42a-0035-412c-a660-54db0fb4480e None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:22.929907 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9733d42a-0035-412c-a660-54db0fb4480e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:22.936982 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9733d42a-0035-412c-a660-54db0fb4480e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:22.938463 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9733d42a-0035-412c-a660-54db0fb4480e None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:22.952998 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-9733d42a-0035-412c-a660-54db0fb4480e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 02:16:22.952998 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:16:22.952998 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:16:22.952998 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:16:22.952998 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:16:22.952998 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:16:22.952998 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:16:22.952998 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:16:22.952998 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:16:22.952998 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:16:22.952998 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:16:22.952998 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:16:22.952998 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 02:16:22.952998 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 02:16:22.952998 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 02:16:22.953960 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 02:16:22.953960 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:16:22.953960 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:16:22.953960 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:16:22.953960 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:16:22.953960 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:16:22.953960 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:16:22.953960 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:16:22.953960 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:16:22.953960 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:16:22.953960 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:16:22.953960 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:16:22.953960 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:16:22.953960 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:16:22.953960 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:16:22.953960 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:16:22.953960 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:16:22.955221 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 02:16:22.955221 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 02:16:22.955221 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 02:16:22.955221 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 02:16:22.955221 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 02:16:22.955221 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 02:16:22.955221 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 02:16:22.955221 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:16:22.959088 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 519/1045] 10.222.0.66 () {56 vars in 1111 bytes} [Mon Jul 18 02:16:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 02:16:22.959889 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9733d42a-0035-412c-a660-54db0fb4480e None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:22.966941 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5e3e9b-d2ab-4519-8179-47b91ddc6ec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:22.967957 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5e3e9b-d2ab-4519-8179-47b91ddc6ec9 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:22.968391 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5e3e9b-d2ab-4519-8179-47b91ddc6ec9 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:22.968827 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5e3e9b-d2ab-4519-8179-47b91ddc6ec9 None admin] PATH_INFO: `/v3/services/image` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:22.969485 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5e3e9b-d2ab-4519-8179-47b91ddc6ec9 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:22.970905 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5e3e9b-d2ab-4519-8179-47b91ddc6ec9 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:22.982662 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-1e5e3e9b-d2ab-4519-8179-47b91ddc6ec9 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 18 02:16:22.982662 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:16:22.982662 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 02:16:22.982662 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 02:16:22.982662 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 02:16:22.982662 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 02:16:22.982662 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 02:16:22.982662 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 02:16:22.982662 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 18 02:16:22.982662 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:16:22.982662 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 02:16:22.982662 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:16:22.982662 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:16:22.982662 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:16:22.982662 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:16:22.982662 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:16:22.982662 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:16:22.983709 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:16:22.983709 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:16:22.983709 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:16:22.983709 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:16:22.983709 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:16:22.983709 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:16:22.983709 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 02:16:22.983709 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 02:16:22.983709 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 02:16:22.983709 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 02:16:22.983709 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:16:22.983709 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:16:22.983709 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:16:22.983709 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:16:22.983709 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:16:22.983709 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:16:22.984620 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:16:22.984620 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:16:22.984620 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:16:22.984620 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:16:22.984620 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:16:22.984620 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:16:22.984620 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:16:22.984620 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:16:22.984620 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:16:22.984620 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:16:22.984620 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 02:16:22.984620 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 02:16:22.984620 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 18 02:16:22.984620 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:16:22.984620 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 527/1046] 10.222.0.66 () {56 vars in 1105 bytes} [Mon Jul 18 02:16:22 2022] GET /identity/v3/services/image => generated 86 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 02:16:22.989558 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5e3e9b-d2ab-4519-8179-47b91ddc6ec9 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:23.070396 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9733d42a-0035-412c-a660-54db0fb4480e None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:23.076375 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:23.099898 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5e3e9b-d2ab-4519-8179-47b91ddc6ec9 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:23.107690 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97731508-0201-489d-8e1e-60ba90680a2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:23.108353 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97731508-0201-489d-8e1e-60ba90680a2e None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:23.108673 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97731508-0201-489d-8e1e-60ba90680a2e None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:23.108962 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97731508-0201-489d-8e1e-60ba90680a2e None admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:23.109793 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97731508-0201-489d-8e1e-60ba90680a2e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:16:23.110125 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97731508-0201-489d-8e1e-60ba90680a2e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:23.111584 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97731508-0201-489d-8e1e-60ba90680a2e None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:23.128979 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 520/1047] 10.222.0.66 () {56 vars in 1148 bytes} [Mon Jul 18 02:16:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 02:16:23.130046 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97731508-0201-489d-8e1e-60ba90680a2e None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:23.140950 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04038ec8-72ae-463a-ad8c-7171bdd6d475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:23.141614 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04038ec8-72ae-463a-ad8c-7171bdd6d475 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:23.142115 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04038ec8-72ae-463a-ad8c-7171bdd6d475 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:23.142247 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04038ec8-72ae-463a-ad8c-7171bdd6d475 None admin] PATH_INFO: `/v3/services` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:23.142905 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04038ec8-72ae-463a-ad8c-7171bdd6d475 None admin] RBAC: Adding query filter params (name=image) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:16:23.143209 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04038ec8-72ae-463a-ad8c-7171bdd6d475 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:23.144570 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04038ec8-72ae-463a-ad8c-7171bdd6d475 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:23.150029 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 528/1048] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 18 02:16:22 2022] GET /identity/v3/services?name=image => generated 122 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 02:16:23.155521 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04038ec8-72ae-463a-ad8c-7171bdd6d475 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:23.187333 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:23.239883 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97731508-0201-489d-8e1e-60ba90680a2e None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:23.261637 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04038ec8-72ae-463a-ad8c-7171bdd6d475 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:23.276888 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857ca4f5-9400-470d-a781-26ec764c273b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:23.277548 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857ca4f5-9400-470d-a781-26ec764c273b None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:23.277835 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857ca4f5-9400-470d-a781-26ec764c273b None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:23.278134 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857ca4f5-9400-470d-a781-26ec764c273b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:23.278981 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857ca4f5-9400-470d-a781-26ec764c273b None admin] RBAC: Adding query filter params (role.id=a1b1364c0cb94959b01bd1c84faa1b9c, scope.project.id=76f7c0ae36ae49b09dcae4c3e9a875f4, user.id=80b138831bc54c7b986d50092d62d4af) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:16:23.279298 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857ca4f5-9400-470d-a781-26ec764c273b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:23.280741 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857ca4f5-9400-470d-a781-26ec764c273b None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:23.286236 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 521/1049] 10.222.0.66 () {56 vars in 1374 bytes} [Mon Jul 18 02:16:23 2022] GET /identity/v3/role_assignments?user.id=80b138831bc54c7b986d50092d62d4af&scope.project.id=76f7c0ae36ae49b09dcae4c3e9a875f4&role.id=a1b1364c0cb94959b01bd1c84faa1b9c => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:16:23.307768 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b80373-3e07-47a2-81db-a76d17170489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:23.308459 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b80373-3e07-47a2-81db-a76d17170489 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:23.308744 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b80373-3e07-47a2-81db-a76d17170489 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:23.309019 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b80373-3e07-47a2-81db-a76d17170489 None admin] PATH_INFO: `/v3/services` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:23.309747 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b80373-3e07-47a2-81db-a76d17170489 None admin] RBAC: Adding query filter params (type=image) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:16:23.310056 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b80373-3e07-47a2-81db-a76d17170489 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:23.311499 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b80373-3e07-47a2-81db-a76d17170489 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:23.317603 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 529/1050] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 18 02:16:23 2022] GET /identity/v3/services?type=image => generated 350 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 02:16:23.323751 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857ca4f5-9400-470d-a781-26ec764c273b None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:23.438173 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857ca4f5-9400-470d-a781-26ec764c273b None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:23.456803 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:23.475442 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2263f9d7-c2ae-448b-baee-7c0ec840d2ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:23.476178 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2263f9d7-c2ae-448b-baee-7c0ec840d2ba None admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:23.476555 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2263f9d7-c2ae-448b-baee-7c0ec840d2ba None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:23.476838 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2263f9d7-c2ae-448b-baee-7c0ec840d2ba None admin] PATH_INFO: `/v3/endpoints` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:23.477595 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2263f9d7-c2ae-448b-baee-7c0ec840d2ba None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:23.479005 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2263f9d7-c2ae-448b-baee-7c0ec840d2ba None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:23.577965 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 522/1051] 10.222.0.66 () {60 vars in 1144 bytes} [Mon Jul 18 02:16:23 2022] POST /identity/v3/endpoints => generated 327 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 02:16:24.719760 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af461721-26b0-4e1c-b211-046e82c2b842 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:24.720289 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af461721-26b0-4e1c-b211-046e82c2b842 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:24.720289 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af461721-26b0-4e1c-b211-046e82c2b842 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:24.721720 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 530/1052] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:16:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:16:24.727441 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108ff4ed-4291-437c-8c19-db17ac7d34e5 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:24.727873 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108ff4ed-4291-437c-8c19-db17ac7d34e5 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:24.728042 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108ff4ed-4291-437c-8c19-db17ac7d34e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:24.744688 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-108ff4ed-4291-437c-8c19-db17ac7d34e5 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:16:24.745868 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-108ff4ed-4291-437c-8c19-db17ac7d34e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:16:24.819815 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 523/1053] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:16:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 02:16:24.842626 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af461721-26b0-4e1c-b211-046e82c2b842 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:24.852899 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:24.945084 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af461721-26b0-4e1c-b211-046e82c2b842 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:24.979411 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6173dd7d-22d2-4697-9805-1943318721a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:24.980193 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6173dd7d-22d2-4697-9805-1943318721a1 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:24.980471 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6173dd7d-22d2-4697-9805-1943318721a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:24.980739 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6173dd7d-22d2-4697-9805-1943318721a1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:24.991158 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6173dd7d-22d2-4697-9805-1943318721a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 02:16:24.998623 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-6173dd7d-22d2-4697-9805-1943318721a1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 02:16:24.998623 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:16:24.998623 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:16:24.998623 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:16:24.998623 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:16:24.998623 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:16:24.998623 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:16:24.998623 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:16:24.998623 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:16:24.998623 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:16:24.998623 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:16:24.998623 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:16:24.998623 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 02:16:24.998623 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 02:16:24.998623 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 02:16:24.998623 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 02:16:24.999567 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 02:16:24.999567 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 02:16:24.999567 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 02:16:24.999567 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 02:16:24.999567 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:16:24.999567 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:16:24.999567 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:16:24.999567 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:16:24.999567 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:16:24.999567 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:16:24.999567 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:16:24.999567 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:16:24.999567 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:16:24.999567 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:16:24.999567 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:16:24.999567 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:16:25.000448 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:16:25.000448 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:16:25.000448 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:16:25.000448 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:16:25.000448 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 02:16:25.000448 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 02:16:25.000448 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 02:16:25.000448 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 02:16:25.000448 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 02:16:25.000448 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 02:16:25.000448 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 02:16:25.000448 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:16:25.002235 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 531/1054] 10.222.0.66 () {56 vars in 1096 bytes} [Mon Jul 18 02:16:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:16:25.005938 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108ff4ed-4291-437c-8c19-db17ac7d34e5 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:25.105161 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:25.109427 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108ff4ed-4291-437c-8c19-db17ac7d34e5 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:25.144670 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df761649-7994-4051-b6e1-668626874b88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:25.145672 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df761649-7994-4051-b6e1-668626874b88 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:25.146159 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df761649-7994-4051-b6e1-668626874b88 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:25.146612 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df761649-7994-4051-b6e1-668626874b88 None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:25.147522 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df761649-7994-4051-b6e1-668626874b88 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:16:25.148682 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df761649-7994-4051-b6e1-668626874b88 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:25.150537 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df761649-7994-4051-b6e1-668626874b88 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:25.159968 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 524/1055] 10.222.0.66 () {56 vars in 1099 bytes} [Mon Jul 18 02:16:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:16:25.166459 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6173dd7d-22d2-4697-9805-1943318721a1 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:25.236746 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:25.271448 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6173dd7d-22d2-4697-9805-1943318721a1 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:25.306839 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39ae8d6-a06b-4e4f-ad33-9d68c4aa5faa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:25.307728 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39ae8d6-a06b-4e4f-ad33-9d68c4aa5faa None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:25.308014 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39ae8d6-a06b-4e4f-ad33-9d68c4aa5faa None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:25.308291 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39ae8d6-a06b-4e4f-ad33-9d68c4aa5faa None admin] PATH_INFO: `/v3/users/3a6ba49b03a849fba6fad0ead46e58b6` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:25.322425 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39ae8d6-a06b-4e4f-ad33-9d68c4aa5faa None admin] RBAC: Authorizing `identity:get_user(user_id=3a6ba49b03a849fba6fad0ead46e58b6)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:25.323884 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39ae8d6-a06b-4e4f-ad33-9d68c4aa5faa None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:25.330502 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 532/1056] 10.222.0.66 () {56 vars in 1177 bytes} [Mon Jul 18 02:16:25 2022] GET /identity/v3/users/3a6ba49b03a849fba6fad0ead46e58b6 => generated 285 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:16:25.331415 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df761649-7994-4051-b6e1-668626874b88 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:25.338232 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21041b4-e469-41dd-ac61-c146b5f90cd6 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:25.338523 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21041b4-e469-41dd-ac61-c146b5f90cd6 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:25.338799 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21041b4-e469-41dd-ac61-c146b5f90cd6 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:25.340455 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 533/1057] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:16:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 02:16:25.345214 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a672d78-10d7-4a42-b21d-df9d52b06dad None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:25.345497 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a672d78-10d7-4a42-b21d-df9d52b06dad None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:25.345769 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a672d78-10d7-4a42-b21d-df9d52b06dad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:25.363167 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-8a672d78-10d7-4a42-b21d-df9d52b06dad None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:16:25.364520 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-8a672d78-10d7-4a42-b21d-df9d52b06dad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:16:25.436801 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 534/1058] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:16:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 02:16:25.444035 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df761649-7994-4051-b6e1-668626874b88 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:25.459524 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a672d78-10d7-4a42-b21d-df9d52b06dad None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:25.479799 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb104aa8-4e37-4ba0-9b4a-3ca9cb892ef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:25.480459 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb104aa8-4e37-4ba0-9b4a-3ca9cb892ef3 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:25.480741 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb104aa8-4e37-4ba0-9b4a-3ca9cb892ef3 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:25.481017 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb104aa8-4e37-4ba0-9b4a-3ca9cb892ef3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:25.510202 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:25.524642 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:25.553593 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb104aa8-4e37-4ba0-9b4a-3ca9cb892ef3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:25.555153 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb104aa8-4e37-4ba0-9b4a-3ca9cb892ef3 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:25.571881 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a672d78-10d7-4a42-b21d-df9d52b06dad None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:25.614682 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10116b8b-b939-4066-b3b2-6cb6c59e838d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:25.615502 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10116b8b-b939-4066-b3b2-6cb6c59e838d None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:25.615787 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10116b8b-b939-4066-b3b2-6cb6c59e838d None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:25.616062 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10116b8b-b939-4066-b3b2-6cb6c59e838d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:25.626758 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10116b8b-b939-4066-b3b2-6cb6c59e838d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 02:16:25.635616 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-10116b8b-b939-4066-b3b2-6cb6c59e838d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 02:16:25.635616 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:16:25.635616 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:16:25.635616 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:16:25.635616 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:16:25.635616 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:16:25.635616 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:16:25.635616 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:16:25.635616 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:16:25.635616 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:16:25.635616 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:16:25.635616 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:16:25.635616 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 02:16:25.635616 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 02:16:25.635616 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 02:16:25.635616 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 02:16:25.636665 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 02:16:25.636665 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 02:16:25.636665 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 02:16:25.636665 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 02:16:25.636665 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:16:25.636665 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:16:25.636665 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:16:25.636665 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:16:25.636665 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:16:25.636665 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:16:25.636665 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:16:25.636665 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:16:25.636665 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:16:25.636665 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:16:25.636665 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:16:25.636665 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:16:25.637529 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:16:25.637529 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:16:25.637529 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:16:25.637529 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:16:25.637529 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 02:16:25.637529 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 02:16:25.637529 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 02:16:25.637529 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 02:16:25.637529 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 02:16:25.637529 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 02:16:25.637529 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 02:16:25.637529 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:16:25.638241 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 535/1059] 10.222.0.66 () {56 vars in 1120 bytes} [Mon Jul 18 02:16:25 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 02:16:25.639556 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f9a658-a00d-40d2-9660-67863fd54647 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:25.640275 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f9a658-a00d-40d2-9660-67863fd54647 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:25.640566 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f9a658-a00d-40d2-9660-67863fd54647 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:25.642284 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 536/1060] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:16:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:16:25.643006 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f9a658-a00d-40d2-9660-67863fd54647 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:25.666251 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 525/1061] 10.222.0.66 () {58 vars in 1282 bytes} [Mon Jul 18 02:16:25 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 18 02:16:25.667632 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da4a793-c2d8-4902-a9e5-425c5194597b None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:25.667921 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da4a793-c2d8-4902-a9e5-425c5194597b None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:25.668201 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da4a793-c2d8-4902-a9e5-425c5194597b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:25.687174 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-8da4a793-c2d8-4902-a9e5-425c5194597b None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:16:25.689053 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-8da4a793-c2d8-4902-a9e5-425c5194597b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:16:25.746852 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:25.748729 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f9a658-a00d-40d2-9660-67863fd54647 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:25.758200 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 526/1062] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:16:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 02:16:25.769693 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da4a793-c2d8-4902-a9e5-425c5194597b None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:25.769827 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:25.787868 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad5a836-a4cf-4c5d-80cc-18f470f186b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:25.789179 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad5a836-a4cf-4c5d-80cc-18f470f186b0 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:25.789455 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad5a836-a4cf-4c5d-80cc-18f470f186b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:25.789727 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad5a836-a4cf-4c5d-80cc-18f470f186b0 None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:25.790544 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad5a836-a4cf-4c5d-80cc-18f470f186b0 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:16:25.790868 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad5a836-a4cf-4c5d-80cc-18f470f186b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:25.792222 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad5a836-a4cf-4c5d-80cc-18f470f186b0 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:25.800951 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 537/1063] 10.222.0.66 () {56 vars in 1115 bytes} [Mon Jul 18 02:16:25 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:16:25.801742 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad5a836-a4cf-4c5d-80cc-18f470f186b0 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:25.860821 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da4a793-c2d8-4902-a9e5-425c5194597b None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:25.894853 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3231c1-d571-4bd2-9ef8-f08a855176f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:25.895763 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3231c1-d571-4bd2-9ef8-f08a855176f2 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:25.896053 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3231c1-d571-4bd2-9ef8-f08a855176f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:25.896332 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3231c1-d571-4bd2-9ef8-f08a855176f2 None admin] PATH_INFO: `/v3/projects/12c1dd6c6ad34aadaa93796c45cd103e` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:25.897514 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3231c1-d571-4bd2-9ef8-f08a855176f2 None admin] RBAC: Authorizing `identity:get_project(project_id=12c1dd6c6ad34aadaa93796c45cd103e)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:25.899515 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3231c1-d571-4bd2-9ef8-f08a855176f2 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:25.901229 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 527/1064] 10.222.0.66 () {56 vars in 1186 bytes} [Mon Jul 18 02:16:25 2022] GET /identity/v3/projects/12c1dd6c6ad34aadaa93796c45cd103e => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:16:25.901332 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad5a836-a4cf-4c5d-80cc-18f470f186b0 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:25.901985 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3231c1-d571-4bd2-9ef8-f08a855176f2 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:25.938626 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1bc23b-8645-4209-a50b-529f30a2a0b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:25.939275 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1bc23b-8645-4209-a50b-529f30a2a0b3 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:25.939559 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1bc23b-8645-4209-a50b-529f30a2a0b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:25.939835 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1bc23b-8645-4209-a50b-529f30a2a0b3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:25.963543 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:25.985108 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:26.005094 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3231c1-d571-4bd2-9ef8-f08a855176f2 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:26.007930 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1bc23b-8645-4209-a50b-529f30a2a0b3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:26.009646 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1bc23b-8645-4209-a50b-529f30a2a0b3 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:26.039866 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8201bd40-324d-4082-b784-b40fa1acb731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:26.040587 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8201bd40-324d-4082-b784-b40fa1acb731 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:26.040877 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8201bd40-324d-4082-b784-b40fa1acb731 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:26.041156 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8201bd40-324d-4082-b784-b40fa1acb731 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:26.052903 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8201bd40-324d-4082-b784-b40fa1acb731 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:26.054388 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8201bd40-324d-4082-b784-b40fa1acb731 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:26.071669 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-8201bd40-324d-4082-b784-b40fa1acb731 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:16:26.071669 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:16:26.071669 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:16:26.071669 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:16:26.071669 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:16:26.071669 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:16:26.071669 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:16:26.071669 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:16:26.071669 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:16:26.071669 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:16:26.071669 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:16:26.071669 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:16:26.071669 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 02:16:26.071669 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 02:16:26.071669 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 02:16:26.072769 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 02:16:26.072769 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:16:26.072769 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:16:26.072769 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:16:26.072769 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:16:26.072769 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:16:26.072769 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:16:26.072769 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:16:26.072769 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:16:26.072769 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:16:26.072769 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:16:26.072769 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:16:26.072769 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:16:26.072769 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:16:26.072769 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:16:26.072769 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:16:26.072769 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:16:26.073688 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 02:16:26.073688 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 02:16:26.073688 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:16:26.073688 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:16:26.073688 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 528/1065] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 18 02:16:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:16:26.073971 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8201bd40-324d-4082-b784-b40fa1acb731 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:26.112121 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 538/1066] 10.222.0.66 () {58 vars in 1282 bytes} [Mon Jul 18 02:16:25 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 02:16:26.112958 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1bc23b-8645-4209-a50b-529f30a2a0b3 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:26.176480 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8201bd40-324d-4082-b784-b40fa1acb731 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:26.186468 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:26.186925 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:26.208540 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1bc23b-8645-4209-a50b-529f30a2a0b3 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:26.211442 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9d59d2-4b38-462a-9351-5694b2703c86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:26.212164 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9d59d2-4b38-462a-9351-5694b2703c86 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:26.212454 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9d59d2-4b38-462a-9351-5694b2703c86 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:26.212733 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9d59d2-4b38-462a-9351-5694b2703c86 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:26.213560 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9d59d2-4b38-462a-9351-5694b2703c86 None admin] RBAC: Adding query filter params (role.id=d4aa3324f0c84887b41d0dc4acf7f30e, scope.project.id=12c1dd6c6ad34aadaa93796c45cd103e, user.id=3a6ba49b03a849fba6fad0ead46e58b6) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:16:26.213875 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9d59d2-4b38-462a-9351-5694b2703c86 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:26.215350 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9d59d2-4b38-462a-9351-5694b2703c86 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:26.221217 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 529/1067] 10.222.0.66 () {56 vars in 1374 bytes} [Mon Jul 18 02:16:26 2022] GET /identity/v3/role_assignments?user.id=3a6ba49b03a849fba6fad0ead46e58b6&scope.project.id=12c1dd6c6ad34aadaa93796c45cd103e&role.id=d4aa3324f0c84887b41d0dc4acf7f30e => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:16:26.221996 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9d59d2-4b38-462a-9351-5694b2703c86 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:26.247413 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6ccbc6-7849-476f-8542-6a6888e116bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:26.249092 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6ccbc6-7849-476f-8542-6a6888e116bc None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:26.250029 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6ccbc6-7849-476f-8542-6a6888e116bc None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:26.251112 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6ccbc6-7849-476f-8542-6a6888e116bc None admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:26.252671 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6ccbc6-7849-476f-8542-6a6888e116bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:16:26.253258 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6ccbc6-7849-476f-8542-6a6888e116bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:26.255527 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6ccbc6-7849-476f-8542-6a6888e116bc None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:26.276433 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 539/1068] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 18 02:16:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:16:26.278152 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6ccbc6-7849-476f-8542-6a6888e116bc None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:26.338446 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9d59d2-4b38-462a-9351-5694b2703c86 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:26.378231 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c8d787-de9a-4239-b10e-beec9a3ffc45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:26.379040 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c8d787-de9a-4239-b10e-beec9a3ffc45 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:26.379326 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c8d787-de9a-4239-b10e-beec9a3ffc45 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:26.379606 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c8d787-de9a-4239-b10e-beec9a3ffc45 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:26.392743 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c8d787-de9a-4239-b10e-beec9a3ffc45 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:26.393832 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6ccbc6-7849-476f-8542-6a6888e116bc None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:26.394257 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c8d787-de9a-4239-b10e-beec9a3ffc45 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:26.412542 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-89c8d787-de9a-4239-b10e-beec9a3ffc45 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:16:26.412542 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:16:26.412542 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:16:26.412542 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:16:26.412542 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:16:26.412542 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:16:26.412542 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:16:26.412542 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:16:26.412542 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:16:26.412542 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:16:26.412542 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:16:26.412542 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:16:26.412542 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 02:16:26.412542 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 02:16:26.412542 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 02:16:26.413929 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 02:16:26.413929 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:16:26.413929 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:16:26.413929 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:16:26.413929 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:16:26.413929 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:16:26.413929 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:16:26.413929 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:16:26.413929 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:16:26.413929 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:16:26.413929 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:16:26.413929 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:16:26.413929 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:16:26.413929 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:16:26.413929 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:16:26.413929 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:16:26.413929 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:16:26.415161 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 02:16:26.415161 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 02:16:26.415161 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:16:26.415161 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:16:26.415161 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 530/1069] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 18 02:16:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:16:26.420503 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c8d787-de9a-4239-b10e-beec9a3ffc45 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:26.436267 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2606dea-5c25-4440-b457-63eda102342b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:26.437319 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2606dea-5c25-4440-b457-63eda102342b None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:26.437608 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2606dea-5c25-4440-b457-63eda102342b None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:26.438385 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2606dea-5c25-4440-b457-63eda102342b None admin] PATH_INFO: `/v3/users/nova` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:26.441782 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:26.448575 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2606dea-5c25-4440-b457-63eda102342b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:26.450040 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2606dea-5c25-4440-b457-63eda102342b None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:26.452888 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:26.465008 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-c2606dea-5c25-4440-b457-63eda102342b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 18 02:16:26.465008 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:16:26.465008 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:16:26.465008 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:16:26.465008 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:16:26.465008 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:16:26.465008 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:16:26.465008 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:16:26.465008 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:16:26.465008 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:16:26.465008 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:16:26.465008 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:16:26.465008 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 02:16:26.465008 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 02:16:26.465008 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 02:16:26.465008 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 02:16:26.466075 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:16:26.466075 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:16:26.466075 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 02:16:26.466075 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 02:16:26.466075 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 02:16:26.466075 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 02:16:26.466075 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:16:26.466075 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:16:26.466075 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:16:26.466075 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:16:26.466075 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:16:26.466075 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:16:26.466075 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:16:26.466075 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:16:26.466075 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:16:26.466075 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:16:26.466075 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:16:26.466075 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:16:26.467087 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:16:26.467087 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:16:26.467087 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 02:16:26.467087 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 02:16:26.467087 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 02:16:26.467087 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 02:16:26.467087 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 02:16:26.467087 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 02:16:26.467087 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 18 02:16:26.467087 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:16:26.472162 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 540/1070] 10.222.0.66 () {56 vars in 1093 bytes} [Mon Jul 18 02:16:26 2022] GET /identity/v3/users/nova => generated 82 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:16:26.473033 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2606dea-5c25-4440-b457-63eda102342b None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:26.534762 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c8d787-de9a-4239-b10e-beec9a3ffc45 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:26.577004 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bc1f3c-44b5-48ef-846e-ba4969013755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:26.577680 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bc1f3c-44b5-48ef-846e-ba4969013755 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:26.577882 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2606dea-5c25-4440-b457-63eda102342b None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:26.577988 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bc1f3c-44b5-48ef-846e-ba4969013755 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:26.578258 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bc1f3c-44b5-48ef-846e-ba4969013755 None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:26.578978 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bc1f3c-44b5-48ef-846e-ba4969013755 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:16:26.579293 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bc1f3c-44b5-48ef-846e-ba4969013755 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:26.581077 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bc1f3c-44b5-48ef-846e-ba4969013755 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:26.602205 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 531/1071] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 18 02:16:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 02:16:26.629084 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7a2ccb-c4c7-4b3f-8faf-07765e704e03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:26.629771 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7a2ccb-c4c7-4b3f-8faf-07765e704e03 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:26.630055 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7a2ccb-c4c7-4b3f-8faf-07765e704e03 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:26.630371 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7a2ccb-c4c7-4b3f-8faf-07765e704e03 None admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:26.631233 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7a2ccb-c4c7-4b3f-8faf-07765e704e03 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:16:26.631547 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7a2ccb-c4c7-4b3f-8faf-07765e704e03 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:26.636675 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7a2ccb-c4c7-4b3f-8faf-07765e704e03 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:26.653774 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 541/1072] 10.222.0.66 () {56 vars in 1133 bytes} [Mon Jul 18 02:16:26 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:16:26.659744 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bc1f3c-44b5-48ef-846e-ba4969013755 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:26.755065 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:26.771933 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bc1f3c-44b5-48ef-846e-ba4969013755 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:26.809382 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffdc272-bdd1-4c7d-952a-4a93e979e178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:26.810210 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffdc272-bdd1-4c7d-952a-4a93e979e178 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:26.810493 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffdc272-bdd1-4c7d-952a-4a93e979e178 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:26.810764 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffdc272-bdd1-4c7d-952a-4a93e979e178 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:26.822330 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffdc272-bdd1-4c7d-952a-4a93e979e178 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:26.823807 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffdc272-bdd1-4c7d-952a-4a93e979e178 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:26.843303 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-8ffdc272-bdd1-4c7d-952a-4a93e979e178 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:16:26.843303 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:16:26.843303 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:16:26.843303 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:16:26.843303 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:16:26.843303 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:16:26.843303 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:16:26.843303 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:16:26.843303 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:16:26.843303 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:16:26.843303 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:16:26.843303 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:16:26.843303 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 02:16:26.843303 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 02:16:26.843303 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 02:16:26.844270 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 02:16:26.844270 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:16:26.844270 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:16:26.844270 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:16:26.844270 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:16:26.844270 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:16:26.844270 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:16:26.844270 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:16:26.844270 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:16:26.844270 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:16:26.844270 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:16:26.844270 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:16:26.844270 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:16:26.844270 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:16:26.844270 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:16:26.844270 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:16:26.844270 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:16:26.845198 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 02:16:26.845198 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 02:16:26.845198 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:16:26.845198 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:16:26.845198 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 532/1073] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 18 02:16:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:16:26.851339 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7a2ccb-c4c7-4b3f-8faf-07765e704e03 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:26.910832 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:26.955273 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7a2ccb-c4c7-4b3f-8faf-07765e704e03 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:26.989351 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b57038e-ad07-4006-a648-3a474870ef7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:26.990011 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b57038e-ad07-4006-a648-3a474870ef7e None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:26.990323 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b57038e-ad07-4006-a648-3a474870ef7e None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:26.990598 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b57038e-ad07-4006-a648-3a474870ef7e None admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:26.991417 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b57038e-ad07-4006-a648-3a474870ef7e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:16:26.991872 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b57038e-ad07-4006-a648-3a474870ef7e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:26.993359 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b57038e-ad07-4006-a648-3a474870ef7e None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:27.004551 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 542/1074] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 18 02:16:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:16:27.010459 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffdc272-bdd1-4c7d-952a-4a93e979e178 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:27.107114 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffdc272-bdd1-4c7d-952a-4a93e979e178 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:27.139865 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a8e1a0-e373-46b3-813e-06809a85b89b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:27.140684 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a8e1a0-e373-46b3-813e-06809a85b89b None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:27.140967 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a8e1a0-e373-46b3-813e-06809a85b89b None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:27.141237 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a8e1a0-e373-46b3-813e-06809a85b89b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:27.148404 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a8e1a0-e373-46b3-813e-06809a85b89b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:27.149889 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a8e1a0-e373-46b3-813e-06809a85b89b None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:27.151615 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:27.164325 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-11a8e1a0-e373-46b3-813e-06809a85b89b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 02:16:27.164325 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:16:27.164325 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:16:27.164325 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:16:27.164325 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:16:27.164325 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:16:27.164325 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:16:27.164325 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:16:27.164325 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:16:27.164325 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:16:27.164325 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:16:27.164325 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:16:27.164325 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 02:16:27.164325 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 02:16:27.164325 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 02:16:27.165363 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 02:16:27.165363 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:16:27.165363 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:16:27.165363 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:16:27.165363 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:16:27.165363 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:16:27.165363 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:16:27.165363 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:16:27.165363 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:16:27.165363 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:16:27.165363 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:16:27.165363 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:16:27.165363 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:16:27.165363 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:16:27.165363 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:16:27.165363 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:16:27.165363 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:16:27.166315 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 02:16:27.166315 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 02:16:27.166315 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 02:16:27.166315 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 02:16:27.166315 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 02:16:27.166315 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 02:16:27.166315 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 02:16:27.166315 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:16:27.166315 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 533/1075] 10.222.0.66 () {56 vars in 1111 bytes} [Mon Jul 18 02:16:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:16:27.171439 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b57038e-ad07-4006-a648-3a474870ef7e None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:27.267924 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b57038e-ad07-4006-a648-3a474870ef7e None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:27.281547 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:27.300918 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3293538c-d859-4564-88a6-13c4d8723194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:27.301574 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3293538c-d859-4564-88a6-13c4d8723194 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:27.301858 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3293538c-d859-4564-88a6-13c4d8723194 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:27.302164 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3293538c-d859-4564-88a6-13c4d8723194 None admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:27.302947 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3293538c-d859-4564-88a6-13c4d8723194 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:16:27.303260 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3293538c-d859-4564-88a6-13c4d8723194 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:27.305226 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3293538c-d859-4564-88a6-13c4d8723194 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:27.317378 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 543/1076] 10.222.0.66 () {56 vars in 1148 bytes} [Mon Jul 18 02:16:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:16:27.323516 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a8e1a0-e373-46b3-813e-06809a85b89b None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:27.423666 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a8e1a0-e373-46b3-813e-06809a85b89b None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:27.458422 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a366a5-509e-4840-b2b6-30469eb031b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:27.459309 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a366a5-509e-4840-b2b6-30469eb031b8 None admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:27.459592 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a366a5-509e-4840-b2b6-30469eb031b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:27.459870 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a366a5-509e-4840-b2b6-30469eb031b8 None admin] PATH_INFO: `/v3/projects/76f7c0ae36ae49b09dcae4c3e9a875f4/users/80b138831bc54c7b986d50092d62d4af/roles/a1b1364c0cb94959b01bd1c84faa1b9c` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:27.462269 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a366a5-509e-4840-b2b6-30469eb031b8 None admin] RBAC: Authorizing `identity:create_grant(project_id=76f7c0ae36ae49b09dcae4c3e9a875f4, user_id=80b138831bc54c7b986d50092d62d4af, role_id=a1b1364c0cb94959b01bd1c84faa1b9c)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:27.463710 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a366a5-509e-4840-b2b6-30469eb031b8 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:27.537274 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 534/1077] 10.222.0.66 () {58 vars in 1439 bytes} [Mon Jul 18 02:16:27 2022] PUT /identity/v3/projects/76f7c0ae36ae49b09dcae4c3e9a875f4/users/80b138831bc54c7b986d50092d62d4af/roles/a1b1364c0cb94959b01bd1c84faa1b9c => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:16:28.255738 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6a2289-f643-4f11-8bf6-e9bf50cff97b None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:28.256311 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6a2289-f643-4f11-8bf6-e9bf50cff97b None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:28.256311 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6a2289-f643-4f11-8bf6-e9bf50cff97b None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:28.258373 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 544/1078] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:16:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:16:28.263830 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d486ce8b-be37-45bc-bf67-3a404a91aecb None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:28.264255 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d486ce8b-be37-45bc-bf67-3a404a91aecb None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:28.264509 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d486ce8b-be37-45bc-bf67-3a404a91aecb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:28.281459 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-d486ce8b-be37-45bc-bf67-3a404a91aecb None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:16:28.282664 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-d486ce8b-be37-45bc-bf67-3a404a91aecb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:16:28.334880 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:28.348857 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 535/1079] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:16:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 02:16:28.370891 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6a2289-f643-4f11-8bf6-e9bf50cff97b None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:28.471098 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6a2289-f643-4f11-8bf6-e9bf50cff97b None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:28.503291 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802a59b4-b3f4-4adc-88a3-6c1b5e5a6878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:28.504067 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802a59b4-b3f4-4adc-88a3-6c1b5e5a6878 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:28.504344 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802a59b4-b3f4-4adc-88a3-6c1b5e5a6878 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:28.504611 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802a59b4-b3f4-4adc-88a3-6c1b5e5a6878 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:28.515242 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802a59b4-b3f4-4adc-88a3-6c1b5e5a6878 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 02:16:28.523564 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-802a59b4-b3f4-4adc-88a3-6c1b5e5a6878 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 02:16:28.523564 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:16:28.523564 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:16:28.523564 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:16:28.523564 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:16:28.523564 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:16:28.523564 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:16:28.523564 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:16:28.523564 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:16:28.523564 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:16:28.523564 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:16:28.523564 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:16:28.523564 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 02:16:28.523564 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 02:16:28.523564 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 02:16:28.523564 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 02:16:28.524525 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 02:16:28.524525 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 02:16:28.524525 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 02:16:28.524525 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 02:16:28.524525 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:16:28.524525 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:16:28.524525 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:16:28.524525 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:16:28.524525 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:16:28.524525 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:16:28.524525 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:16:28.524525 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:16:28.524525 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:16:28.524525 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:16:28.524525 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:16:28.524525 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:16:28.525399 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:16:28.525399 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:16:28.525399 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:16:28.525399 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:16:28.525399 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 02:16:28.525399 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 02:16:28.525399 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 02:16:28.525399 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 02:16:28.525399 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 02:16:28.525399 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 02:16:28.525399 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 02:16:28.525399 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:16:28.525399 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 545/1080] 10.222.0.66 () {56 vars in 1096 bytes} [Mon Jul 18 02:16:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:16:28.530826 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d486ce8b-be37-45bc-bf67-3a404a91aecb None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:28.628190 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d486ce8b-be37-45bc-bf67-3a404a91aecb None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:28.664331 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0a5803-0b2d-446b-89c8-f59774b8c196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:28.664994 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0a5803-0b2d-446b-89c8-f59774b8c196 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:28.665275 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0a5803-0b2d-446b-89c8-f59774b8c196 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:28.665550 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0a5803-0b2d-446b-89c8-f59774b8c196 None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:28.666312 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0a5803-0b2d-446b-89c8-f59774b8c196 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:16:28.666617 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0a5803-0b2d-446b-89c8-f59774b8c196 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:28.667982 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0a5803-0b2d-446b-89c8-f59774b8c196 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:28.676167 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 536/1081] 10.222.0.66 () {56 vars in 1099 bytes} [Mon Jul 18 02:16:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:16:28.682365 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802a59b4-b3f4-4adc-88a3-6c1b5e5a6878 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:28.717361 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:28.784273 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802a59b4-b3f4-4adc-88a3-6c1b5e5a6878 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:28.818055 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72614c9-3cfe-4d1c-aeb9-c7d88b4a03c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:28.818937 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72614c9-3cfe-4d1c-aeb9-c7d88b4a03c8 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:28.819222 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72614c9-3cfe-4d1c-aeb9-c7d88b4a03c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:28.819550 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72614c9-3cfe-4d1c-aeb9-c7d88b4a03c8 None admin] PATH_INFO: `/v3/users/3a6ba49b03a849fba6fad0ead46e58b6` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:28.820730 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72614c9-3cfe-4d1c-aeb9-c7d88b4a03c8 None admin] RBAC: Authorizing `identity:get_user(user_id=3a6ba49b03a849fba6fad0ead46e58b6)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:28.822210 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72614c9-3cfe-4d1c-aeb9-c7d88b4a03c8 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:28.824016 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 546/1082] 10.222.0.66 () {56 vars in 1177 bytes} [Mon Jul 18 02:16:28 2022] GET /identity/v3/users/3a6ba49b03a849fba6fad0ead46e58b6 => generated 285 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:16:28.829725 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0a5803-0b2d-446b-89c8-f59774b8c196 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:28.878782 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:28.928211 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0a5803-0b2d-446b-89c8-f59774b8c196 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:28.960747 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa51004d-a9b3-4b7d-ad71-1628c795d4d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:28.961612 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa51004d-a9b3-4b7d-ad71-1628c795d4d9 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:28.961898 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa51004d-a9b3-4b7d-ad71-1628c795d4d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:28.962212 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa51004d-a9b3-4b7d-ad71-1628c795d4d9 None admin] PATH_INFO: `/v3/projects/12c1dd6c6ad34aadaa93796c45cd103e` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:28.963560 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa51004d-a9b3-4b7d-ad71-1628c795d4d9 None admin] RBAC: Authorizing `identity:get_project(project_id=12c1dd6c6ad34aadaa93796c45cd103e)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:28.965018 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa51004d-a9b3-4b7d-ad71-1628c795d4d9 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:28.966794 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 537/1083] 10.222.0.66 () {56 vars in 1186 bytes} [Mon Jul 18 02:16:28 2022] GET /identity/v3/projects/12c1dd6c6ad34aadaa93796c45cd103e => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:16:28.972880 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72614c9-3cfe-4d1c-aeb9-c7d88b4a03c8 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:29.072186 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72614c9-3cfe-4d1c-aeb9-c7d88b4a03c8 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:29.073638 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:29.106734 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b75fb8d-d760-4008-8925-eacbdd07dcf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:29.107686 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b75fb8d-d760-4008-8925-eacbdd07dcf3 None admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:29.108137 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b75fb8d-d760-4008-8925-eacbdd07dcf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:29.108458 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b75fb8d-d760-4008-8925-eacbdd07dcf3 None admin] PATH_INFO: `/v3/projects/12c1dd6c6ad34aadaa93796c45cd103e/users/3a6ba49b03a849fba6fad0ead46e58b6/roles/d4aa3324f0c84887b41d0dc4acf7f30e` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:29.110773 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b75fb8d-d760-4008-8925-eacbdd07dcf3 None admin] RBAC: Authorizing `identity:create_grant(project_id=12c1dd6c6ad34aadaa93796c45cd103e, user_id=3a6ba49b03a849fba6fad0ead46e58b6, role_id=d4aa3324f0c84887b41d0dc4acf7f30e)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:29.112214 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b75fb8d-d760-4008-8925-eacbdd07dcf3 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:29.175939 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 547/1084] 10.222.0.66 () {58 vars in 1439 bytes} [Mon Jul 18 02:16:28 2022] PUT /identity/v3/projects/12c1dd6c6ad34aadaa93796c45cd103e/users/3a6ba49b03a849fba6fad0ead46e58b6/roles/d4aa3324f0c84887b41d0dc4acf7f30e => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:16:29.552380 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb0e7f7-c906-423f-82bd-ae5cf550fe5a None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:29.552950 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb0e7f7-c906-423f-82bd-ae5cf550fe5a None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:29.552950 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb0e7f7-c906-423f-82bd-ae5cf550fe5a None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:29.555074 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 538/1085] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:16:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:16:29.560459 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8b5454-77c1-471c-a51f-e7c37d22a55a None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:29.560933 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8b5454-77c1-471c-a51f-e7c37d22a55a None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:29.561098 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8b5454-77c1-471c-a51f-e7c37d22a55a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:29.578262 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-df8b5454-77c1-471c-a51f-e7c37d22a55a None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:16:29.579431 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-df8b5454-77c1-471c-a51f-e7c37d22a55a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:16:29.648588 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 548/1086] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:16:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 02:16:29.671398 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb0e7f7-c906-423f-82bd-ae5cf550fe5a None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:29.773338 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb0e7f7-c906-423f-82bd-ae5cf550fe5a None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:29.790848 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:29.811423 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95855784-b201-48bf-83cc-d799a6dd3fba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:29.811423 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95855784-b201-48bf-83cc-d799a6dd3fba None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:29.811423 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95855784-b201-48bf-83cc-d799a6dd3fba None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:29.811423 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95855784-b201-48bf-83cc-d799a6dd3fba None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:29.820373 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95855784-b201-48bf-83cc-d799a6dd3fba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 02:16:29.828694 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-95855784-b201-48bf-83cc-d799a6dd3fba None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 02:16:29.828694 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:16:29.828694 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:16:29.828694 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:16:29.828694 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:16:29.828694 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:16:29.828694 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:16:29.828694 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:16:29.828694 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:16:29.828694 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:16:29.828694 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:16:29.828694 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:16:29.828694 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 02:16:29.828694 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 02:16:29.828694 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 02:16:29.828694 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 02:16:29.829880 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 02:16:29.829880 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 02:16:29.829880 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 02:16:29.829880 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 02:16:29.829880 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:16:29.829880 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:16:29.829880 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:16:29.829880 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:16:29.829880 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:16:29.829880 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:16:29.829880 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:16:29.829880 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:16:29.829880 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:16:29.829880 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:16:29.829880 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:16:29.829880 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:16:29.830800 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:16:29.830800 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:16:29.830800 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:16:29.830800 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:16:29.830800 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 02:16:29.830800 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 02:16:29.830800 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 02:16:29.830800 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 02:16:29.830800 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 02:16:29.830800 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 02:16:29.830800 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 02:16:29.830800 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:16:29.830800 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 539/1087] 10.222.0.66 () {56 vars in 1120 bytes} [Mon Jul 18 02:16:29 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:16:29.836683 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8b5454-77c1-471c-a51f-e7c37d22a55a None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:29.927250 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:29.936282 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8b5454-77c1-471c-a51f-e7c37d22a55a None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:29.970656 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5f1af8-5c39-4f26-b531-4a7b05eb948e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:29.971318 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5f1af8-5c39-4f26-b531-4a7b05eb948e None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:29.971604 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5f1af8-5c39-4f26-b531-4a7b05eb948e None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:29.971930 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5f1af8-5c39-4f26-b531-4a7b05eb948e None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:29.972668 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5f1af8-5c39-4f26-b531-4a7b05eb948e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:16:29.972969 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5f1af8-5c39-4f26-b531-4a7b05eb948e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:29.974364 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5f1af8-5c39-4f26-b531-4a7b05eb948e None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:29.982454 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 549/1088] 10.222.0.66 () {56 vars in 1115 bytes} [Mon Jul 18 02:16:29 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:16:29.987998 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95855784-b201-48bf-83cc-d799a6dd3fba None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:30.086391 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95855784-b201-48bf-83cc-d799a6dd3fba None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:30.118206 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b1c39c-7ec7-423f-b7ff-1d7b0be0deb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:30.118980 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b1c39c-7ec7-423f-b7ff-1d7b0be0deb6 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:30.119258 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b1c39c-7ec7-423f-b7ff-1d7b0be0deb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:30.119528 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b1c39c-7ec7-423f-b7ff-1d7b0be0deb6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:30.129706 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b1c39c-7ec7-423f-b7ff-1d7b0be0deb6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:30.131187 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b1c39c-7ec7-423f-b7ff-1d7b0be0deb6 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:30.148612 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-88b1c39c-7ec7-423f-b7ff-1d7b0be0deb6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:16:30.148612 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:16:30.148612 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:16:30.148612 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:16:30.148612 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:16:30.148612 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:16:30.148612 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:16:30.148612 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:16:30.148612 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:16:30.148612 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:16:30.148612 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:16:30.148612 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:16:30.148612 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 02:16:30.148612 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 02:16:30.148612 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 02:16:30.149517 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 02:16:30.149517 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:16:30.149517 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:16:30.149517 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:16:30.149517 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:16:30.149517 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:16:30.149517 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:16:30.149517 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:16:30.149517 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:16:30.149517 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:16:30.149517 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:16:30.149517 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:16:30.149517 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:16:30.149517 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:16:30.149517 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:16:30.149517 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:16:30.149517 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:16:30.150503 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 02:16:30.150503 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 02:16:30.150503 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:16:30.150503 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:16:30.150503 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 540/1089] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 18 02:16:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 02:16:30.155595 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5f1af8-5c39-4f26-b531-4a7b05eb948e None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:30.254153 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5f1af8-5c39-4f26-b531-4a7b05eb948e None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:30.289704 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a099eed8-b8f0-4417-8391-9cbfed7e3b62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:30.290391 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a099eed8-b8f0-4417-8391-9cbfed7e3b62 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:30.290679 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a099eed8-b8f0-4417-8391-9cbfed7e3b62 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:30.290957 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a099eed8-b8f0-4417-8391-9cbfed7e3b62 None admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:30.291677 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a099eed8-b8f0-4417-8391-9cbfed7e3b62 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:16:30.291983 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a099eed8-b8f0-4417-8391-9cbfed7e3b62 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:30.293359 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a099eed8-b8f0-4417-8391-9cbfed7e3b62 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:30.305560 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 550/1090] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 18 02:16:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:16:30.311179 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b1c39c-7ec7-423f-b7ff-1d7b0be0deb6 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:30.342287 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:30.454224 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b1c39c-7ec7-423f-b7ff-1d7b0be0deb6 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:30.488121 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db77e66d-e3d6-4d9c-ae75-9ab35f90c88c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:30.488910 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db77e66d-e3d6-4d9c-ae75-9ab35f90c88c None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:30.489196 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db77e66d-e3d6-4d9c-ae75-9ab35f90c88c None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:30.489470 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db77e66d-e3d6-4d9c-ae75-9ab35f90c88c None admin] PATH_INFO: `/v3/users/nova` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:30.496769 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db77e66d-e3d6-4d9c-ae75-9ab35f90c88c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:30.498254 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db77e66d-e3d6-4d9c-ae75-9ab35f90c88c None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:30.512648 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-db77e66d-e3d6-4d9c-ae75-9ab35f90c88c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 18 02:16:30.512648 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:16:30.512648 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:16:30.512648 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:16:30.512648 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:16:30.512648 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:16:30.512648 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:16:30.512648 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:16:30.512648 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:16:30.512648 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:16:30.512648 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:16:30.512648 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:16:30.512648 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 02:16:30.512648 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 02:16:30.512648 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 02:16:30.512648 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 02:16:30.513670 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:16:30.513670 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:16:30.513670 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 02:16:30.513670 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 02:16:30.513670 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 02:16:30.513670 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 02:16:30.513670 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:16:30.513670 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:16:30.513670 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:16:30.513670 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:16:30.513670 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:16:30.513670 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:16:30.513670 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:16:30.513670 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:16:30.513670 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:16:30.513670 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:16:30.513670 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:16:30.513670 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:16:30.514665 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:16:30.514665 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:16:30.514665 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 02:16:30.514665 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 02:16:30.514665 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 02:16:30.514665 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 02:16:30.514665 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 02:16:30.514665 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 02:16:30.514665 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 18 02:16:30.514665 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:16:30.514665 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 541/1091] 10.222.0.66 () {56 vars in 1093 bytes} [Mon Jul 18 02:16:30 2022] GET /identity/v3/users/nova => generated 82 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 02:16:30.519737 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a099eed8-b8f0-4417-8391-9cbfed7e3b62 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:30.535506 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:30.620309 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a099eed8-b8f0-4417-8391-9cbfed7e3b62 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:30.654371 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1314f498-4c81-46f8-b650-fb551d842440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:30.655035 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1314f498-4c81-46f8-b650-fb551d842440 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:30.655321 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1314f498-4c81-46f8-b650-fb551d842440 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:30.655601 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1314f498-4c81-46f8-b650-fb551d842440 None admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:30.656430 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1314f498-4c81-46f8-b650-fb551d842440 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:16:30.656744 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1314f498-4c81-46f8-b650-fb551d842440 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:30.658192 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1314f498-4c81-46f8-b650-fb551d842440 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:30.675417 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 551/1092] 10.222.0.66 () {56 vars in 1133 bytes} [Mon Jul 18 02:16:30 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:16:30.681100 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db77e66d-e3d6-4d9c-ae75-9ab35f90c88c None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:30.775769 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:30.791465 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db77e66d-e3d6-4d9c-ae75-9ab35f90c88c None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:30.827131 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da501bc3-d93c-4118-963d-59b5868af0b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:30.827783 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da501bc3-d93c-4118-963d-59b5868af0b1 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:30.828068 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da501bc3-d93c-4118-963d-59b5868af0b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:30.828346 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da501bc3-d93c-4118-963d-59b5868af0b1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:30.894699 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da501bc3-d93c-4118-963d-59b5868af0b1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:30.896072 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da501bc3-d93c-4118-963d-59b5868af0b1 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:30.982817 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:30.993649 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 542/1093] 10.222.0.66 () {58 vars in 1282 bytes} [Mon Jul 18 02:16:30 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 02:16:30.999641 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1314f498-4c81-46f8-b650-fb551d842440 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:31.068036 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:31.097439 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1314f498-4c81-46f8-b650-fb551d842440 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:31.130997 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf674c7-ba2d-4183-8fee-da69e2c31f77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:31.131952 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf674c7-ba2d-4183-8fee-da69e2c31f77 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:31.132241 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf674c7-ba2d-4183-8fee-da69e2c31f77 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:31.132522 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf674c7-ba2d-4183-8fee-da69e2c31f77 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:31.142472 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf674c7-ba2d-4183-8fee-da69e2c31f77 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:31.143948 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf674c7-ba2d-4183-8fee-da69e2c31f77 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:31.163687 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-8cf674c7-ba2d-4183-8fee-da69e2c31f77 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:16:31.163687 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:16:31.163687 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:16:31.163687 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:16:31.163687 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:16:31.163687 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:16:31.163687 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:16:31.163687 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:16:31.163687 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:16:31.163687 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:16:31.163687 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:16:31.163687 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:16:31.163687 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 02:16:31.163687 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 02:16:31.163687 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 02:16:31.164835 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 02:16:31.164835 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:16:31.164835 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:16:31.164835 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:16:31.164835 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:16:31.164835 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:16:31.164835 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:16:31.164835 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:16:31.164835 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:16:31.164835 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:16:31.164835 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:16:31.164835 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:16:31.164835 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:16:31.164835 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:16:31.164835 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:16:31.164835 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:16:31.164835 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:16:31.165790 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 02:16:31.165790 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 02:16:31.165790 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:16:31.165790 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:16:31.165790 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 552/1094] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 18 02:16:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:16:31.171142 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da501bc3-d93c-4118-963d-59b5868af0b1 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:31.171899 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd432fc4-47d3-4586-99af-a3d52fce155c None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:31.172188 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd432fc4-47d3-4586-99af-a3d52fce155c None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:31.172465 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd432fc4-47d3-4586-99af-a3d52fce155c None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:31.173787 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 553/1095] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:16:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 02:16:31.178751 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6460080f-bb55-4575-9e96-4846ebd4ed25 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:31.179036 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6460080f-bb55-4575-9e96-4846ebd4ed25 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:31.179314 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6460080f-bb55-4575-9e96-4846ebd4ed25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:31.195285 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-6460080f-bb55-4575-9e96-4846ebd4ed25 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:16:31.196469 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-6460080f-bb55-4575-9e96-4846ebd4ed25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:16:31.262364 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 554/1096] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:16:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 02:16:31.273170 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da501bc3-d93c-4118-963d-59b5868af0b1 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:31.286051 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6460080f-bb55-4575-9e96-4846ebd4ed25 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:31.308357 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055572e1-3170-4012-b553-4b8ccde74c4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:31.309023 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055572e1-3170-4012-b553-4b8ccde74c4b None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:31.309304 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055572e1-3170-4012-b553-4b8ccde74c4b None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:31.309581 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055572e1-3170-4012-b553-4b8ccde74c4b None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:31.310329 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055572e1-3170-4012-b553-4b8ccde74c4b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:16:31.310640 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055572e1-3170-4012-b553-4b8ccde74c4b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:31.312018 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055572e1-3170-4012-b553-4b8ccde74c4b None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:31.323201 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 543/1097] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 18 02:16:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:16:31.328752 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055572e1-3170-4012-b553-4b8ccde74c4b None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:31.341981 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:31.378301 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:31.385614 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6460080f-bb55-4575-9e96-4846ebd4ed25 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:31.420284 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe1f66b-2504-44ae-baf6-642674912477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:31.421000 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe1f66b-2504-44ae-baf6-642674912477 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:31.421282 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe1f66b-2504-44ae-baf6-642674912477 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:31.421553 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe1f66b-2504-44ae-baf6-642674912477 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:31.428203 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055572e1-3170-4012-b553-4b8ccde74c4b None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:31.431588 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe1f66b-2504-44ae-baf6-642674912477 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 02:16:31.439204 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-bbe1f66b-2504-44ae-baf6-642674912477 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 02:16:31.439204 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:16:31.439204 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:16:31.439204 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:16:31.439204 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:16:31.439204 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:16:31.439204 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:16:31.439204 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:16:31.439204 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:16:31.439204 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:16:31.439204 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:16:31.439204 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:16:31.439204 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 02:16:31.439204 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 02:16:31.439204 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 02:16:31.439204 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 02:16:31.440233 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 02:16:31.440233 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 02:16:31.440233 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 02:16:31.440233 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 02:16:31.440233 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:16:31.440233 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:16:31.440233 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:16:31.440233 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:16:31.440233 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:16:31.440233 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:16:31.440233 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:16:31.440233 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:16:31.440233 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:16:31.440233 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:16:31.440233 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:16:31.440233 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:16:31.441121 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:16:31.441121 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:16:31.441121 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:16:31.441121 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:16:31.441121 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 02:16:31.441121 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 02:16:31.441121 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 02:16:31.441121 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 02:16:31.441121 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 02:16:31.441121 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 02:16:31.441121 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 02:16:31.441121 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:16:31.441121 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 555/1098] 10.222.0.66 () {56 vars in 1096 bytes} [Mon Jul 18 02:16:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 02:16:31.447799 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe1f66b-2504-44ae-baf6-642674912477 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:31.462634 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ae088d-b36d-46a5-84fe-a709db05a71f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:31.463455 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ae088d-b36d-46a5-84fe-a709db05a71f None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:31.463742 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ae088d-b36d-46a5-84fe-a709db05a71f None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:31.464032 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ae088d-b36d-46a5-84fe-a709db05a71f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:31.470643 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ae088d-b36d-46a5-84fe-a709db05a71f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:31.472117 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ae088d-b36d-46a5-84fe-a709db05a71f None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:31.486922 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-06ae088d-b36d-46a5-84fe-a709db05a71f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 02:16:31.486922 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:16:31.486922 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:16:31.486922 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:16:31.486922 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:16:31.486922 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:16:31.486922 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:16:31.486922 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:16:31.486922 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:16:31.486922 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:16:31.486922 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:16:31.486922 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:16:31.486922 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 02:16:31.486922 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 02:16:31.486922 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 02:16:31.488029 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 02:16:31.488029 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:16:31.488029 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:16:31.488029 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:16:31.488029 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:16:31.488029 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:16:31.488029 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:16:31.488029 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:16:31.488029 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:16:31.488029 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:16:31.488029 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:16:31.488029 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:16:31.488029 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:16:31.488029 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:16:31.488029 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:16:31.488029 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:16:31.488029 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:16:31.489026 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 02:16:31.489026 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 02:16:31.489026 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 02:16:31.489026 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 02:16:31.489026 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 02:16:31.489026 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 02:16:31.489026 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 02:16:31.489026 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:16:31.489026 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 544/1099] 10.222.0.66 () {56 vars in 1111 bytes} [Mon Jul 18 02:16:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 02:16:31.495427 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ae088d-b36d-46a5-84fe-a709db05a71f None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:31.550366 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe1f66b-2504-44ae-baf6-642674912477 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:31.564569 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:31.583806 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d4c811-c030-40a7-afa5-02dc615eca69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:31.584466 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d4c811-c030-40a7-afa5-02dc615eca69 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:31.584784 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d4c811-c030-40a7-afa5-02dc615eca69 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:31.585057 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d4c811-c030-40a7-afa5-02dc615eca69 None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:31.585802 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d4c811-c030-40a7-afa5-02dc615eca69 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:16:31.586188 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d4c811-c030-40a7-afa5-02dc615eca69 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:31.587574 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d4c811-c030-40a7-afa5-02dc615eca69 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:31.594714 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ae088d-b36d-46a5-84fe-a709db05a71f None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:31.596208 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 556/1100] 10.222.0.66 () {56 vars in 1099 bytes} [Mon Jul 18 02:16:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 02:16:31.602642 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d4c811-c030-40a7-afa5-02dc615eca69 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:31.610274 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:31.633776 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3745304c-c1dd-4a55-9c5d-210fbdd2bb30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:31.634470 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3745304c-c1dd-4a55-9c5d-210fbdd2bb30 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:31.634755 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3745304c-c1dd-4a55-9c5d-210fbdd2bb30 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:31.635033 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3745304c-c1dd-4a55-9c5d-210fbdd2bb30 None admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:31.635810 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3745304c-c1dd-4a55-9c5d-210fbdd2bb30 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:16:31.636120 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3745304c-c1dd-4a55-9c5d-210fbdd2bb30 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:31.637565 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3745304c-c1dd-4a55-9c5d-210fbdd2bb30 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:31.649259 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 545/1101] 10.222.0.66 () {56 vars in 1148 bytes} [Mon Jul 18 02:16:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 02:16:31.655305 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3745304c-c1dd-4a55-9c5d-210fbdd2bb30 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:31.704803 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d4c811-c030-40a7-afa5-02dc615eca69 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:31.738646 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6989d31a-c9b7-4b01-9ba2-d101ed98c615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:31.739546 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6989d31a-c9b7-4b01-9ba2-d101ed98c615 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:31.739822 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6989d31a-c9b7-4b01-9ba2-d101ed98c615 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:31.740099 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6989d31a-c9b7-4b01-9ba2-d101ed98c615 None admin] PATH_INFO: `/v3/users/3a6ba49b03a849fba6fad0ead46e58b6` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:31.741373 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6989d31a-c9b7-4b01-9ba2-d101ed98c615 None admin] RBAC: Authorizing `identity:get_user(user_id=3a6ba49b03a849fba6fad0ead46e58b6)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:31.742835 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6989d31a-c9b7-4b01-9ba2-d101ed98c615 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:31.744497 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 557/1102] 10.222.0.66 () {56 vars in 1177 bytes} [Mon Jul 18 02:16:31 2022] GET /identity/v3/users/3a6ba49b03a849fba6fad0ead46e58b6 => generated 285 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:16:31.750411 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6989d31a-c9b7-4b01-9ba2-d101ed98c615 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:31.754972 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3745304c-c1dd-4a55-9c5d-210fbdd2bb30 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:31.789610 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3ef424-920e-4edd-8471-e8e6686fcb95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:31.790300 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3ef424-920e-4edd-8471-e8e6686fcb95 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:31.790584 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3ef424-920e-4edd-8471-e8e6686fcb95 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:31.790859 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3ef424-920e-4edd-8471-e8e6686fcb95 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:31.791683 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3ef424-920e-4edd-8471-e8e6686fcb95 None admin] RBAC: Adding query filter params (role.id=a1b1364c0cb94959b01bd1c84faa1b9c, scope.project.id=76f7c0ae36ae49b09dcae4c3e9a875f4, user.id=80b138831bc54c7b986d50092d62d4af) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:16:31.791993 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3ef424-920e-4edd-8471-e8e6686fcb95 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:31.793431 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3ef424-920e-4edd-8471-e8e6686fcb95 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:31.796556 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:31.808630 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 546/1103] 10.222.0.66 () {56 vars in 1374 bytes} [Mon Jul 18 02:16:31 2022] GET /identity/v3/role_assignments?user.id=80b138831bc54c7b986d50092d62d4af&scope.project.id=76f7c0ae36ae49b09dcae4c3e9a875f4&role.id=a1b1364c0cb94959b01bd1c84faa1b9c => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:16:31.856008 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6989d31a-c9b7-4b01-9ba2-d101ed98c615 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:31.891742 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfae4199-01a5-414f-9b89-f955c335f173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:31.892407 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfae4199-01a5-414f-9b89-f955c335f173 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:31.892693 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfae4199-01a5-414f-9b89-f955c335f173 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:31.892972 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfae4199-01a5-414f-9b89-f955c335f173 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:31.965055 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfae4199-01a5-414f-9b89-f955c335f173 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:31.966483 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfae4199-01a5-414f-9b89-f955c335f173 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:32.010301 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:32.068533 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 558/1104] 10.222.0.66 () {58 vars in 1282 bytes} [Mon Jul 18 02:16:31 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 02:16:32.074734 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3ef424-920e-4edd-8471-e8e6686fcb95 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:32.113153 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:32.184229 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3ef424-920e-4edd-8471-e8e6686fcb95 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:32.220236 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f81e4d-96a2-43e1-a635-41eecec31d41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:32.220236 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f81e4d-96a2-43e1-a635-41eecec31d41 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:32.220529 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f81e4d-96a2-43e1-a635-41eecec31d41 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:32.220715 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f81e4d-96a2-43e1-a635-41eecec31d41 None admin] PATH_INFO: `/v3/projects/12c1dd6c6ad34aadaa93796c45cd103e` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:32.221897 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f81e4d-96a2-43e1-a635-41eecec31d41 None admin] RBAC: Authorizing `identity:get_project(project_id=12c1dd6c6ad34aadaa93796c45cd103e)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:32.223375 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f81e4d-96a2-43e1-a635-41eecec31d41 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:32.225098 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 547/1105] 10.222.0.66 () {56 vars in 1186 bytes} [Mon Jul 18 02:16:32 2022] GET /identity/v3/projects/12c1dd6c6ad34aadaa93796c45cd103e => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:16:32.232042 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfae4199-01a5-414f-9b89-f955c335f173 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:32.334475 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfae4199-01a5-414f-9b89-f955c335f173 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:32.370005 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc11545e-8017-49b6-a4a0-af001643438e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:32.370694 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc11545e-8017-49b6-a4a0-af001643438e None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:32.370976 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc11545e-8017-49b6-a4a0-af001643438e None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:32.371246 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc11545e-8017-49b6-a4a0-af001643438e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:32.372108 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc11545e-8017-49b6-a4a0-af001643438e None admin] RBAC: Adding query filter params (role.id=d4aa3324f0c84887b41d0dc4acf7f30e, scope.project.id=12c1dd6c6ad34aadaa93796c45cd103e, user.id=3a6ba49b03a849fba6fad0ead46e58b6) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:16:32.372413 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc11545e-8017-49b6-a4a0-af001643438e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:32.373841 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc11545e-8017-49b6-a4a0-af001643438e None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:32.380155 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 559/1106] 10.222.0.66 () {56 vars in 1374 bytes} [Mon Jul 18 02:16:32 2022] GET /identity/v3/role_assignments?user.id=3a6ba49b03a849fba6fad0ead46e58b6&scope.project.id=12c1dd6c6ad34aadaa93796c45cd103e&role.id=d4aa3324f0c84887b41d0dc4acf7f30e => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:16:34.396554 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef78f627-6a0b-4c75-beab-5c2a36003e3a None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:34.397195 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef78f627-6a0b-4c75-beab-5c2a36003e3a None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:34.397444 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef78f627-6a0b-4c75-beab-5c2a36003e3a None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:34.398655 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 548/1107] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:16:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:16:34.403837 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60b2ad5-f9cd-402a-bcae-4060e02bd77d None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:34.404234 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60b2ad5-f9cd-402a-bcae-4060e02bd77d None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:34.404394 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60b2ad5-f9cd-402a-bcae-4060e02bd77d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:34.423938 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-d60b2ad5-f9cd-402a-bcae-4060e02bd77d None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:16:34.425116 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-d60b2ad5-f9cd-402a-bcae-4060e02bd77d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:16:34.427097 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:34.491696 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 560/1108] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:16:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 02:16:34.515808 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef78f627-6a0b-4c75-beab-5c2a36003e3a None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:34.617877 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef78f627-6a0b-4c75-beab-5c2a36003e3a None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:34.619473 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:34.651923 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f718f945-84ef-475f-be28-8bccfd9ebd5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:34.652644 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f718f945-84ef-475f-be28-8bccfd9ebd5a None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:34.652931 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f718f945-84ef-475f-be28-8bccfd9ebd5a None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:34.653211 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f718f945-84ef-475f-be28-8bccfd9ebd5a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:34.654413 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f718f945-84ef-475f-be28-8bccfd9ebd5a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:34.655855 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f718f945-84ef-475f-be28-8bccfd9ebd5a None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:34.657487 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 549/1109] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 18 02:16:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:16:34.664030 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60b2ad5-f9cd-402a-bcae-4060e02bd77d None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:34.758951 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60b2ad5-f9cd-402a-bcae-4060e02bd77d None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:34.796144 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c269b3c6-e008-4366-a016-70324733ba32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:34.796809 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c269b3c6-e008-4366-a016-70324733ba32 None admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:34.797095 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c269b3c6-e008-4366-a016-70324733ba32 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:34.797370 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c269b3c6-e008-4366-a016-70324733ba32 None admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:34.798115 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c269b3c6-e008-4366-a016-70324733ba32 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:34.799557 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c269b3c6-e008-4366-a016-70324733ba32 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:34.807958 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:34.926832 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 561/1110] 10.222.0.66 () {60 vars in 1132 bytes} [Mon Jul 18 02:16:34 2022] POST /identity/v3/users => generated 286 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 02:16:36.953773 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0527d5f-c4f1-4d60-b7e3-0dc160833adf None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:36.954364 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0527d5f-c4f1-4d60-b7e3-0dc160833adf None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:36.954364 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0527d5f-c4f1-4d60-b7e3-0dc160833adf None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:36.955722 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 550/1111] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:16:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:16:36.961445 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953b6915-5727-40bf-a1cd-400efa40402f None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:36.961880 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953b6915-5727-40bf-a1cd-400efa40402f None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:36.961997 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953b6915-5727-40bf-a1cd-400efa40402f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:36.978896 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-953b6915-5727-40bf-a1cd-400efa40402f None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:16:36.980068 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-953b6915-5727-40bf-a1cd-400efa40402f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:16:37.044516 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 562/1112] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:16:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 02:16:37.067373 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0527d5f-c4f1-4d60-b7e3-0dc160833adf None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:37.167695 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0527d5f-c4f1-4d60-b7e3-0dc160833adf None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:37.206726 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d63efc-c4ca-47d0-a50c-7c94db6be5e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:37.207507 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d63efc-c4ca-47d0-a50c-7c94db6be5e3 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:37.207784 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d63efc-c4ca-47d0-a50c-7c94db6be5e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:37.208056 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d63efc-c4ca-47d0-a50c-7c94db6be5e3 None admin] PATH_INFO: `/v3/roles/a5b895323e7b4f0db2491abe9ad57ee9` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:37.210122 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d63efc-c4ca-47d0-a50c-7c94db6be5e3 None admin] RBAC: Authorizing `identity:get_role(role_id=a5b895323e7b4f0db2491abe9ad57ee9)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:37.211492 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d63efc-c4ca-47d0-a50c-7c94db6be5e3 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:37.212864 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 551/1113] 10.222.0.66 () {56 vars in 1177 bytes} [Mon Jul 18 02:16:37 2022] GET /identity/v3/roles/a5b895323e7b4f0db2491abe9ad57ee9 => generated 223 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:16:37.218865 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953b6915-5727-40bf-a1cd-400efa40402f None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:37.313006 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953b6915-5727-40bf-a1cd-400efa40402f None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:37.335823 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:37.345847 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38980281-25b6-4028-ab43-324b631634fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:37.346727 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38980281-25b6-4028-ab43-324b631634fe None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:37.347007 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38980281-25b6-4028-ab43-324b631634fe None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:37.347278 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38980281-25b6-4028-ab43-324b631634fe None admin] PATH_INFO: `/v3/users/9f70614f690043c78cf98bcec2037fd7` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:37.360814 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38980281-25b6-4028-ab43-324b631634fe None admin] RBAC: Authorizing `identity:get_user(user_id=9f70614f690043c78cf98bcec2037fd7)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:37.362277 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38980281-25b6-4028-ab43-324b631634fe None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:37.363974 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 563/1114] 10.222.0.66 () {56 vars in 1177 bytes} [Mon Jul 18 02:16:37 2022] GET /identity/v3/users/9f70614f690043c78cf98bcec2037fd7 => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:16:37.369820 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d63efc-c4ca-47d0-a50c-7c94db6be5e3 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:37.409503 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:37.467386 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d63efc-c4ca-47d0-a50c-7c94db6be5e3 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:37.500639 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6481fec-a2a0-4dff-a8ae-ac3c42510825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:37.501287 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6481fec-a2a0-4dff-a8ae-ac3c42510825 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:37.501574 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6481fec-a2a0-4dff-a8ae-ac3c42510825 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:37.501853 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6481fec-a2a0-4dff-a8ae-ac3c42510825 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:37.572072 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6481fec-a2a0-4dff-a8ae-ac3c42510825 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:37.573612 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6481fec-a2a0-4dff-a8ae-ac3c42510825 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:37.612366 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:37.677460 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 552/1115] 10.222.0.66 () {58 vars in 1282 bytes} [Mon Jul 18 02:16:37 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 02:16:37.683595 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38980281-25b6-4028-ab43-324b631634fe None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:37.788405 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38980281-25b6-4028-ab43-324b631634fe None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:37.825980 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99180e62-9d83-40d5-a18e-b070e6d8ad44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:37.826992 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99180e62-9d83-40d5-a18e-b070e6d8ad44 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:37.827517 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99180e62-9d83-40d5-a18e-b070e6d8ad44 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:37.828051 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99180e62-9d83-40d5-a18e-b070e6d8ad44 None admin] PATH_INFO: `/v3/projects/12c1dd6c6ad34aadaa93796c45cd103e` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:37.829336 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99180e62-9d83-40d5-a18e-b070e6d8ad44 None admin] RBAC: Authorizing `identity:get_project(project_id=12c1dd6c6ad34aadaa93796c45cd103e)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:37.830824 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99180e62-9d83-40d5-a18e-b070e6d8ad44 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:37.832548 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 564/1116] 10.222.0.66 () {56 vars in 1186 bytes} [Mon Jul 18 02:16:37 2022] GET /identity/v3/projects/12c1dd6c6ad34aadaa93796c45cd103e => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:16:37.838882 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6481fec-a2a0-4dff-a8ae-ac3c42510825 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:37.948109 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6481fec-a2a0-4dff-a8ae-ac3c42510825 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:37.982906 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6c168a-00a2-4d71-b7b6-95e6e4c8a304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:37.983552 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6c168a-00a2-4d71-b7b6-95e6e4c8a304 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:37.983821 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6c168a-00a2-4d71-b7b6-95e6e4c8a304 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:37.984095 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6c168a-00a2-4d71-b7b6-95e6e4c8a304 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:37.984916 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6c168a-00a2-4d71-b7b6-95e6e4c8a304 None admin] RBAC: Adding query filter params (role.id=a5b895323e7b4f0db2491abe9ad57ee9, scope.project.id=12c1dd6c6ad34aadaa93796c45cd103e, user.id=9f70614f690043c78cf98bcec2037fd7) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:16:37.985231 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6c168a-00a2-4d71-b7b6-95e6e4c8a304 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:37.986697 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6c168a-00a2-4d71-b7b6-95e6e4c8a304 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:37.993124 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 553/1117] 10.222.0.66 () {56 vars in 1374 bytes} [Mon Jul 18 02:16:37 2022] GET /identity/v3/role_assignments?user.id=9f70614f690043c78cf98bcec2037fd7&scope.project.id=12c1dd6c6ad34aadaa93796c45cd103e&role.id=a5b895323e7b4f0db2491abe9ad57ee9 => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:16:39.996298 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddc876c-6051-4c69-9949-8e54dc111a36 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:39.996939 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddc876c-6051-4c69-9949-8e54dc111a36 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:39.996939 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddc876c-6051-4c69-9949-8e54dc111a36 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:39.998258 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 565/1118] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:16:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:16:40.003509 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cf449b-9abf-4d82-9b72-c9bf9d78927e None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:40.003967 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cf449b-9abf-4d82-9b72-c9bf9d78927e None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:40.004070 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cf449b-9abf-4d82-9b72-c9bf9d78927e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:40.009729 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:40.021488 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-59cf449b-9abf-4d82-9b72-c9bf9d78927e None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:16:40.024177 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-59cf449b-9abf-4d82-9b72-c9bf9d78927e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:16:40.090495 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 554/1119] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:16:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 02:16:40.112482 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddc876c-6051-4c69-9949-8e54dc111a36 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:40.161952 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:40.210923 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddc876c-6051-4c69-9949-8e54dc111a36 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:40.243188 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c189fd17-dedc-4aee-a3f6-8a0dd37cb623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:40.244016 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c189fd17-dedc-4aee-a3f6-8a0dd37cb623 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:40.244349 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c189fd17-dedc-4aee-a3f6-8a0dd37cb623 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:40.244625 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c189fd17-dedc-4aee-a3f6-8a0dd37cb623 None admin] PATH_INFO: `/v3/roles/a5b895323e7b4f0db2491abe9ad57ee9` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:40.246065 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c189fd17-dedc-4aee-a3f6-8a0dd37cb623 None admin] RBAC: Authorizing `identity:get_role(role_id=a5b895323e7b4f0db2491abe9ad57ee9)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:40.247503 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c189fd17-dedc-4aee-a3f6-8a0dd37cb623 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:40.248795 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 566/1120] 10.222.0.66 () {56 vars in 1177 bytes} [Mon Jul 18 02:16:40 2022] GET /identity/v3/roles/a5b895323e7b4f0db2491abe9ad57ee9 => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:16:40.254747 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cf449b-9abf-4d82-9b72-c9bf9d78927e None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:40.351440 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cf449b-9abf-4d82-9b72-c9bf9d78927e None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:40.380006 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:40.384842 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a8d16c-8be2-4075-9c6a-f971981eee31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:40.385706 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a8d16c-8be2-4075-9c6a-f971981eee31 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:40.385988 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a8d16c-8be2-4075-9c6a-f971981eee31 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:40.386294 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a8d16c-8be2-4075-9c6a-f971981eee31 None admin] PATH_INFO: `/v3/users/9f70614f690043c78cf98bcec2037fd7` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:40.387466 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a8d16c-8be2-4075-9c6a-f971981eee31 None admin] RBAC: Authorizing `identity:get_user(user_id=9f70614f690043c78cf98bcec2037fd7)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:40.388906 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a8d16c-8be2-4075-9c6a-f971981eee31 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:40.390568 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 555/1121] 10.222.0.66 () {56 vars in 1177 bytes} [Mon Jul 18 02:16:40 2022] GET /identity/v3/users/9f70614f690043c78cf98bcec2037fd7 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:16:40.401674 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c189fd17-dedc-4aee-a3f6-8a0dd37cb623 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:40.494846 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c189fd17-dedc-4aee-a3f6-8a0dd37cb623 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:40.511657 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:40.527867 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d27e75-2040-4929-800e-d829aefd39fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:40.532551 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d27e75-2040-4929-800e-d829aefd39fb None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:40.532551 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d27e75-2040-4929-800e-d829aefd39fb None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:40.532551 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d27e75-2040-4929-800e-d829aefd39fb None admin] PATH_INFO: `/v3/projects/12c1dd6c6ad34aadaa93796c45cd103e` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:40.532551 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d27e75-2040-4929-800e-d829aefd39fb None admin] RBAC: Authorizing `identity:get_project(project_id=12c1dd6c6ad34aadaa93796c45cd103e)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:40.532551 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d27e75-2040-4929-800e-d829aefd39fb None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:40.533592 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 567/1122] 10.222.0.66 () {56 vars in 1186 bytes} [Mon Jul 18 02:16:40 2022] GET /identity/v3/projects/12c1dd6c6ad34aadaa93796c45cd103e => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:16:40.540474 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a8d16c-8be2-4075-9c6a-f971981eee31 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:40.638928 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a8d16c-8be2-4075-9c6a-f971981eee31 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:40.672015 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3f5d2d-919d-4d4e-b1bb-28e2b0b7116e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:40.672892 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3f5d2d-919d-4d4e-b1bb-28e2b0b7116e None admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:40.673174 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3f5d2d-919d-4d4e-b1bb-28e2b0b7116e None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:40.673452 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3f5d2d-919d-4d4e-b1bb-28e2b0b7116e None admin] PATH_INFO: `/v3/projects/12c1dd6c6ad34aadaa93796c45cd103e/users/9f70614f690043c78cf98bcec2037fd7/roles/a5b895323e7b4f0db2491abe9ad57ee9` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:40.675635 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3f5d2d-919d-4d4e-b1bb-28e2b0b7116e None admin] RBAC: Authorizing `identity:create_grant(project_id=12c1dd6c6ad34aadaa93796c45cd103e, user_id=9f70614f690043c78cf98bcec2037fd7, role_id=a5b895323e7b4f0db2491abe9ad57ee9)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:40.677063 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3f5d2d-919d-4d4e-b1bb-28e2b0b7116e None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:40.802676 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 556/1123] 10.222.0.66 () {58 vars in 1439 bytes} [Mon Jul 18 02:16:40 2022] PUT /identity/v3/projects/12c1dd6c6ad34aadaa93796c45cd103e/users/9f70614f690043c78cf98bcec2037fd7/roles/a5b895323e7b4f0db2491abe9ad57ee9 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:16:42.794138 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a28b05d-0466-453c-961c-47b78205cf46 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:42.794673 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a28b05d-0466-453c-961c-47b78205cf46 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:42.794673 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a28b05d-0466-453c-961c-47b78205cf46 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:42.796081 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 568/1124] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:16:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:16:42.801325 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09152bbd-514a-4930-bcd8-181d94b9a79c None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:42.801768 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09152bbd-514a-4930-bcd8-181d94b9a79c None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:42.801874 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09152bbd-514a-4930-bcd8-181d94b9a79c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:42.818829 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-09152bbd-514a-4930-bcd8-181d94b9a79c None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:16:42.820027 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-09152bbd-514a-4930-bcd8-181d94b9a79c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:16:42.873909 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:42.884787 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 557/1125] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:16:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 02:16:42.907614 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a28b05d-0466-453c-961c-47b78205cf46 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:43.006426 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a28b05d-0466-453c-961c-47b78205cf46 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:43.039375 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3e0328-4fff-4c55-9c17-2dbd23c8352e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:43.040212 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3e0328-4fff-4c55-9c17-2dbd23c8352e None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:43.040498 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3e0328-4fff-4c55-9c17-2dbd23c8352e None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:43.040773 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3e0328-4fff-4c55-9c17-2dbd23c8352e None admin] PATH_INFO: `/v3/roles/a5b895323e7b4f0db2491abe9ad57ee9` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:43.042272 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3e0328-4fff-4c55-9c17-2dbd23c8352e None admin] RBAC: Authorizing `identity:get_role(role_id=a5b895323e7b4f0db2491abe9ad57ee9)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:43.043918 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3e0328-4fff-4c55-9c17-2dbd23c8352e None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:43.045288 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 569/1126] 10.222.0.66 () {56 vars in 1177 bytes} [Mon Jul 18 02:16:42 2022] GET /identity/v3/roles/a5b895323e7b4f0db2491abe9ad57ee9 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:16:43.051140 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09152bbd-514a-4930-bcd8-181d94b9a79c None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:43.146893 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09152bbd-514a-4930-bcd8-181d94b9a79c None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:43.180272 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2642ee-90d2-48d4-976d-b595341f5dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:43.181127 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2642ee-90d2-48d4-976d-b595341f5dc5 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:43.181409 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2642ee-90d2-48d4-976d-b595341f5dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:43.181682 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2642ee-90d2-48d4-976d-b595341f5dc5 None admin] PATH_INFO: `/v3/users/9f70614f690043c78cf98bcec2037fd7` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:43.182882 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2642ee-90d2-48d4-976d-b595341f5dc5 None admin] RBAC: Authorizing `identity:get_user(user_id=9f70614f690043c78cf98bcec2037fd7)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:43.184313 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2642ee-90d2-48d4-976d-b595341f5dc5 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:43.185927 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 558/1127] 10.222.0.66 () {56 vars in 1177 bytes} [Mon Jul 18 02:16:43 2022] GET /identity/v3/users/9f70614f690043c78cf98bcec2037fd7 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:16:43.191630 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3e0328-4fff-4c55-9c17-2dbd23c8352e None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:43.245739 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:43.290754 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3e0328-4fff-4c55-9c17-2dbd23c8352e None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:43.324266 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1934a442-8832-45e8-b315-0cffae13d075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:43.324899 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1934a442-8832-45e8-b315-0cffae13d075 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:43.325179 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1934a442-8832-45e8-b315-0cffae13d075 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:43.325450 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1934a442-8832-45e8-b315-0cffae13d075 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:43.390142 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1934a442-8832-45e8-b315-0cffae13d075 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:43.391582 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1934a442-8832-45e8-b315-0cffae13d075 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:43.442938 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:43.486817 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 570/1128] 10.222.0.66 () {58 vars in 1282 bytes} [Mon Jul 18 02:16:43 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 02:16:43.493319 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2642ee-90d2-48d4-976d-b595341f5dc5 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:43.550284 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:43.592455 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2642ee-90d2-48d4-976d-b595341f5dc5 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:43.625966 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb81ac13-775b-4b6c-a5a8-9351fd90a527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:43.626840 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb81ac13-775b-4b6c-a5a8-9351fd90a527 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:43.627119 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb81ac13-775b-4b6c-a5a8-9351fd90a527 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:43.627385 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb81ac13-775b-4b6c-a5a8-9351fd90a527 None admin] PATH_INFO: `/v3/projects/12c1dd6c6ad34aadaa93796c45cd103e` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:43.628537 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb81ac13-775b-4b6c-a5a8-9351fd90a527 None admin] RBAC: Authorizing `identity:get_project(project_id=12c1dd6c6ad34aadaa93796c45cd103e)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:43.629964 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb81ac13-775b-4b6c-a5a8-9351fd90a527 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:43.631834 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 559/1129] 10.222.0.66 () {56 vars in 1186 bytes} [Mon Jul 18 02:16:43 2022] GET /identity/v3/projects/12c1dd6c6ad34aadaa93796c45cd103e => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:16:43.637899 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1934a442-8832-45e8-b315-0cffae13d075 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:43.735509 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1934a442-8832-45e8-b315-0cffae13d075 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:43.767973 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9a0c05-856c-4d83-be9b-f9d6fed1ddd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:43.768610 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9a0c05-856c-4d83-be9b-f9d6fed1ddd5 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:43.768896 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9a0c05-856c-4d83-be9b-f9d6fed1ddd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:43.769172 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9a0c05-856c-4d83-be9b-f9d6fed1ddd5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:43.769989 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9a0c05-856c-4d83-be9b-f9d6fed1ddd5 None admin] RBAC: Adding query filter params (role.id=a5b895323e7b4f0db2491abe9ad57ee9, scope.project.id=12c1dd6c6ad34aadaa93796c45cd103e, user.id=9f70614f690043c78cf98bcec2037fd7) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:16:43.770333 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9a0c05-856c-4d83-be9b-f9d6fed1ddd5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:43.772301 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9a0c05-856c-4d83-be9b-f9d6fed1ddd5 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:43.777989 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 571/1130] 10.222.0.66 () {56 vars in 1374 bytes} [Mon Jul 18 02:16:43 2022] GET /identity/v3/role_assignments?user.id=9f70614f690043c78cf98bcec2037fd7&scope.project.id=12c1dd6c6ad34aadaa93796c45cd103e&role.id=a5b895323e7b4f0db2491abe9ad57ee9 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:16:45.774212 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b31292-b052-45f1-b9d4-894624cfd7ea None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:45.774768 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b31292-b052-45f1-b9d4-894624cfd7ea None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:45.774768 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b31292-b052-45f1-b9d4-894624cfd7ea None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:45.776141 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 560/1131] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:16:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:16:45.781341 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7804d2b3-d806-4ce6-b61b-cca40245e6c6 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:45.781778 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7804d2b3-d806-4ce6-b61b-cca40245e6c6 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:45.781873 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7804d2b3-d806-4ce6-b61b-cca40245e6c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:45.798815 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-7804d2b3-d806-4ce6-b61b-cca40245e6c6 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:16:45.799982 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-7804d2b3-d806-4ce6-b61b-cca40245e6c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:16:45.813227 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:45.865070 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 572/1132] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:16:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 02:16:45.887718 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b31292-b052-45f1-b9d4-894624cfd7ea None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:45.985906 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b31292-b052-45f1-b9d4-894624cfd7ea None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:46.010863 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:46.019193 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed39bfa4-c025-40e7-a0bd-8f49bd7d09a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:46.022139 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed39bfa4-c025-40e7-a0bd-8f49bd7d09a0 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:46.022139 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed39bfa4-c025-40e7-a0bd-8f49bd7d09a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:46.022139 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed39bfa4-c025-40e7-a0bd-8f49bd7d09a0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:46.022493 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed39bfa4-c025-40e7-a0bd-8f49bd7d09a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:46.023731 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed39bfa4-c025-40e7-a0bd-8f49bd7d09a0 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:46.025359 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 561/1133] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 18 02:16:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:16:46.031151 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7804d2b3-d806-4ce6-b61b-cca40245e6c6 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:46.127216 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7804d2b3-d806-4ce6-b61b-cca40245e6c6 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:46.159947 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06642d4-be7c-4e91-ba28-d04c5e5b093b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:46.160591 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06642d4-be7c-4e91-ba28-d04c5e5b093b None admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:46.160879 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06642d4-be7c-4e91-ba28-d04c5e5b093b None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:46.161706 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06642d4-be7c-4e91-ba28-d04c5e5b093b None admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:46.162621 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06642d4-be7c-4e91-ba28-d04c5e5b093b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:46.164047 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06642d4-be7c-4e91-ba28-d04c5e5b093b None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:46.272637 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:46.279054 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 573/1134] 10.222.0.66 () {60 vars in 1141 bytes} [Mon Jul 18 02:16:46 2022] POST /identity/v3/projects => generated 311 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:16:48.286776 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd64206-5b8c-4bec-a6ed-17dfd34e9208 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:48.287319 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd64206-5b8c-4bec-a6ed-17dfd34e9208 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:48.287433 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd64206-5b8c-4bec-a6ed-17dfd34e9208 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:48.288817 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 562/1135] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:16:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:16:48.294453 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f274864b-17a4-41bc-a37f-053b4eabbe5c None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:48.294876 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f274864b-17a4-41bc-a37f-053b4eabbe5c None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:48.295014 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f274864b-17a4-41bc-a37f-053b4eabbe5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:48.312137 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-f274864b-17a4-41bc-a37f-053b4eabbe5c None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:16:48.313314 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-f274864b-17a4-41bc-a37f-053b4eabbe5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:16:48.378433 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 574/1136] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:16:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 02:16:48.406658 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd64206-5b8c-4bec-a6ed-17dfd34e9208 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:48.506235 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd64206-5b8c-4bec-a6ed-17dfd34e9208 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:48.539567 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaf2da5-1eda-44df-97ed-c353578d40ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:48.540288 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaf2da5-1eda-44df-97ed-c353578d40ff None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:48.540574 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaf2da5-1eda-44df-97ed-c353578d40ff None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:48.540848 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaf2da5-1eda-44df-97ed-c353578d40ff None admin] PATH_INFO: `/v3/domains/default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:48.542021 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbaf2da5-1eda-44df-97ed-c353578d40ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:48.543736 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbaf2da5-1eda-44df-97ed-c353578d40ff None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:48.545355 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 563/1137] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 18 02:16:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:16:48.551189 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f274864b-17a4-41bc-a37f-053b4eabbe5c None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:48.646952 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f274864b-17a4-41bc-a37f-053b4eabbe5c None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:48.674243 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:48.680039 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd6cabb-0e3b-41bd-8983-76f35c94043e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:48.680699 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd6cabb-0e3b-41bd-8983-76f35c94043e None admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:48.680983 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd6cabb-0e3b-41bd-8983-76f35c94043e None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:48.681259 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd6cabb-0e3b-41bd-8983-76f35c94043e None admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:48.681972 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd6cabb-0e3b-41bd-8983-76f35c94043e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:48.683435 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd6cabb-0e3b-41bd-8983-76f35c94043e None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:48.791402 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 575/1138] 10.222.0.66 () {60 vars in 1132 bytes} [Mon Jul 18 02:16:48 2022] POST /identity/v3/users => generated 286 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:16:50.844646 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7035dc63-6c67-4998-aa9a-02926d5fbf6c None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:50.845199 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7035dc63-6c67-4998-aa9a-02926d5fbf6c None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:50.845199 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7035dc63-6c67-4998-aa9a-02926d5fbf6c None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:50.846619 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 564/1139] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:16:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:16:50.852414 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44feb6d9-56d5-41ac-b1e0-25a292857482 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:50.852822 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44feb6d9-56d5-41ac-b1e0-25a292857482 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:50.853008 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44feb6d9-56d5-41ac-b1e0-25a292857482 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:50.869523 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-44feb6d9-56d5-41ac-b1e0-25a292857482 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:16:50.870658 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-44feb6d9-56d5-41ac-b1e0-25a292857482 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:16:50.934911 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 576/1140] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:16:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 02:16:50.958507 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7035dc63-6c67-4998-aa9a-02926d5fbf6c None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:51.019799 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:51.058930 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7035dc63-6c67-4998-aa9a-02926d5fbf6c None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:51.091384 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54c2020-460d-4d7a-90b8-160ca735cc5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:51.092161 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54c2020-460d-4d7a-90b8-160ca735cc5a None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:51.092439 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54c2020-460d-4d7a-90b8-160ca735cc5a None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:51.092705 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54c2020-460d-4d7a-90b8-160ca735cc5a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:51.103165 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54c2020-460d-4d7a-90b8-160ca735cc5a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 02:16:51.110663 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-d54c2020-460d-4d7a-90b8-160ca735cc5a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 02:16:51.110663 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:16:51.110663 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:16:51.110663 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:16:51.110663 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:16:51.110663 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:16:51.110663 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:16:51.110663 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:16:51.110663 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:16:51.110663 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:16:51.110663 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:16:51.110663 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:16:51.110663 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 02:16:51.110663 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 02:16:51.110663 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 02:16:51.110663 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 02:16:51.111835 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 02:16:51.111835 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 02:16:51.111835 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 02:16:51.111835 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 02:16:51.111835 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:16:51.111835 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:16:51.111835 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:16:51.111835 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:16:51.111835 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:16:51.111835 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:16:51.111835 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:16:51.111835 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:16:51.111835 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:16:51.111835 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:16:51.111835 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:16:51.111835 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:16:51.113076 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:16:51.113076 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:16:51.113076 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:16:51.113076 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:16:51.113076 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 02:16:51.113076 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 02:16:51.113076 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 02:16:51.113076 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 02:16:51.113076 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 02:16:51.113076 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 02:16:51.113076 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 02:16:51.113076 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:16:51.113076 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 565/1141] 10.222.0.66 () {56 vars in 1096 bytes} [Mon Jul 18 02:16:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:16:51.119022 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44feb6d9-56d5-41ac-b1e0-25a292857482 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:51.215315 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44feb6d9-56d5-41ac-b1e0-25a292857482 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:51.230865 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:51.248050 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8639b44-52c7-462b-a0c6-42d35d8756e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:51.248705 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8639b44-52c7-462b-a0c6-42d35d8756e7 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:51.248990 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8639b44-52c7-462b-a0c6-42d35d8756e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:51.249269 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8639b44-52c7-462b-a0c6-42d35d8756e7 None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:51.250016 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8639b44-52c7-462b-a0c6-42d35d8756e7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:16:51.250357 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8639b44-52c7-462b-a0c6-42d35d8756e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:51.251720 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8639b44-52c7-462b-a0c6-42d35d8756e7 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:51.259654 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 577/1142] 10.222.0.66 () {56 vars in 1099 bytes} [Mon Jul 18 02:16:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:16:51.265430 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54c2020-460d-4d7a-90b8-160ca735cc5a None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:51.368963 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54c2020-460d-4d7a-90b8-160ca735cc5a None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:51.391408 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:51.403324 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfd0725-2075-4e6d-9b4f-5ee588969b19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:51.404191 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfd0725-2075-4e6d-9b4f-5ee588969b19 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:51.404475 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfd0725-2075-4e6d-9b4f-5ee588969b19 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:51.404753 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfd0725-2075-4e6d-9b4f-5ee588969b19 None admin] PATH_INFO: `/v3/users/ebe70be515e944fd8fac78a193a6e9a3` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:51.418557 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfd0725-2075-4e6d-9b4f-5ee588969b19 None admin] RBAC: Authorizing `identity:get_user(user_id=ebe70be515e944fd8fac78a193a6e9a3)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:51.420103 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfd0725-2075-4e6d-9b4f-5ee588969b19 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:51.421770 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 566/1143] 10.222.0.66 () {56 vars in 1177 bytes} [Mon Jul 18 02:16:51 2022] GET /identity/v3/users/ebe70be515e944fd8fac78a193a6e9a3 => generated 286 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:16:51.427845 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8639b44-52c7-462b-a0c6-42d35d8756e7 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:51.526991 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8639b44-52c7-462b-a0c6-42d35d8756e7 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:51.560613 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b2ea5c-0d7d-459c-aab6-2e893d5d6229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:51.561253 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b2ea5c-0d7d-459c-aab6-2e893d5d6229 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:51.561539 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b2ea5c-0d7d-459c-aab6-2e893d5d6229 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:51.561817 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b2ea5c-0d7d-459c-aab6-2e893d5d6229 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:51.607887 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:51.630301 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b2ea5c-0d7d-459c-aab6-2e893d5d6229 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:51.631758 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b2ea5c-0d7d-459c-aab6-2e893d5d6229 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:51.736778 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 578/1144] 10.222.0.66 () {58 vars in 1282 bytes} [Mon Jul 18 02:16:51 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 02:16:51.744812 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfd0725-2075-4e6d-9b4f-5ee588969b19 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:51.857695 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfd0725-2075-4e6d-9b4f-5ee588969b19 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:51.891770 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab18faf-317b-4114-a1f0-3ac24de6524f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:51.892638 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab18faf-317b-4114-a1f0-3ac24de6524f None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:51.892924 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab18faf-317b-4114-a1f0-3ac24de6524f None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:51.893201 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab18faf-317b-4114-a1f0-3ac24de6524f None admin] PATH_INFO: `/v3/projects/149fc3c6e00d499ab731067be3c4c77f` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:51.894394 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab18faf-317b-4114-a1f0-3ac24de6524f None admin] RBAC: Authorizing `identity:get_project(project_id=149fc3c6e00d499ab731067be3c4c77f)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:51.895925 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab18faf-317b-4114-a1f0-3ac24de6524f None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:51.897636 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 567/1145] 10.222.0.66 () {56 vars in 1186 bytes} [Mon Jul 18 02:16:51 2022] GET /identity/v3/projects/149fc3c6e00d499ab731067be3c4c77f => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:16:51.903988 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b2ea5c-0d7d-459c-aab6-2e893d5d6229 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:51.980542 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:52.000789 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b2ea5c-0d7d-459c-aab6-2e893d5d6229 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:52.033916 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2504e83c-fee0-46b9-8c81-39f529397c50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:52.034591 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2504e83c-fee0-46b9-8c81-39f529397c50 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:52.034879 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2504e83c-fee0-46b9-8c81-39f529397c50 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:52.035161 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2504e83c-fee0-46b9-8c81-39f529397c50 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:52.036063 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2504e83c-fee0-46b9-8c81-39f529397c50 None admin] RBAC: Adding query filter params (role.id=d4aa3324f0c84887b41d0dc4acf7f30e, scope.project.id=149fc3c6e00d499ab731067be3c4c77f, user.id=ebe70be515e944fd8fac78a193a6e9a3) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:16:52.036372 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2504e83c-fee0-46b9-8c81-39f529397c50 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:52.037798 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2504e83c-fee0-46b9-8c81-39f529397c50 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:52.043403 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 579/1146] 10.222.0.66 () {56 vars in 1374 bytes} [Mon Jul 18 02:16:51 2022] GET /identity/v3/role_assignments?user.id=ebe70be515e944fd8fac78a193a6e9a3&scope.project.id=149fc3c6e00d499ab731067be3c4c77f&role.id=d4aa3324f0c84887b41d0dc4acf7f30e => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:16:54.022001 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f55416-ed8c-4913-b943-e902fc902918 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:54.022589 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f55416-ed8c-4913-b943-e902fc902918 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:54.022694 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f55416-ed8c-4913-b943-e902fc902918 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:54.024012 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 568/1147] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:16:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 02:16:54.029407 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e6da0c-2155-4e54-8be2-56c6b385e129 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:54.029839 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e6da0c-2155-4e54-8be2-56c6b385e129 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:54.029956 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e6da0c-2155-4e54-8be2-56c6b385e129 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:54.046568 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-80e6da0c-2155-4e54-8be2-56c6b385e129 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:16:54.047732 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-80e6da0c-2155-4e54-8be2-56c6b385e129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:16:54.111662 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 580/1148] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:16:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 02:16:54.133555 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f55416-ed8c-4913-b943-e902fc902918 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:54.182728 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:54.233192 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f55416-ed8c-4913-b943-e902fc902918 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:54.265869 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e63805-ffc8-4cc9-bbbc-b01ca317b15d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:54.266679 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e63805-ffc8-4cc9-bbbc-b01ca317b15d None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:54.266962 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e63805-ffc8-4cc9-bbbc-b01ca317b15d None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:54.267238 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e63805-ffc8-4cc9-bbbc-b01ca317b15d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:54.277762 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e63805-ffc8-4cc9-bbbc-b01ca317b15d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 02:16:54.285352 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-24e63805-ffc8-4cc9-bbbc-b01ca317b15d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 02:16:54.285352 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:16:54.285352 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:16:54.285352 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:16:54.285352 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:16:54.285352 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:16:54.285352 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:16:54.285352 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:16:54.285352 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:16:54.285352 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:16:54.285352 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:16:54.285352 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:16:54.285352 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 02:16:54.285352 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 02:16:54.285352 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 02:16:54.285352 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 02:16:54.286521 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 02:16:54.286521 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 02:16:54.286521 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 02:16:54.286521 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 02:16:54.286521 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:16:54.286521 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:16:54.286521 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:16:54.286521 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:16:54.286521 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:16:54.286521 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:16:54.286521 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:16:54.286521 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:16:54.286521 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:16:54.286521 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:16:54.286521 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:16:54.286521 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:16:54.287728 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:16:54.287728 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:16:54.287728 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:16:54.287728 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:16:54.287728 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 02:16:54.287728 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 02:16:54.287728 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 02:16:54.287728 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 02:16:54.287728 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 02:16:54.287728 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 02:16:54.287728 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 02:16:54.287728 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:16:54.287728 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 569/1149] 10.222.0.66 () {56 vars in 1096 bytes} [Mon Jul 18 02:16:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:16:54.292741 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e6da0c-2155-4e54-8be2-56c6b385e129 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:54.359596 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:54.388083 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e6da0c-2155-4e54-8be2-56c6b385e129 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:54.421259 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b615df9-d5f4-4b14-bb82-fa89bfe843e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:54.421914 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b615df9-d5f4-4b14-bb82-fa89bfe843e2 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:54.422248 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b615df9-d5f4-4b14-bb82-fa89bfe843e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:54.422525 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b615df9-d5f4-4b14-bb82-fa89bfe843e2 None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:54.423258 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b615df9-d5f4-4b14-bb82-fa89bfe843e2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:16:54.423565 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b615df9-d5f4-4b14-bb82-fa89bfe843e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:54.424944 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b615df9-d5f4-4b14-bb82-fa89bfe843e2 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:54.433335 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 581/1150] 10.222.0.66 () {56 vars in 1099 bytes} [Mon Jul 18 02:16:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:16:54.439140 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e63805-ffc8-4cc9-bbbc-b01ca317b15d None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:54.537493 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e63805-ffc8-4cc9-bbbc-b01ca317b15d None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:54.546189 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:54.570994 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac7ba17-c8b8-4c9a-be1f-b95f43f0eb8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:54.571850 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac7ba17-c8b8-4c9a-be1f-b95f43f0eb8e None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:54.572213 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac7ba17-c8b8-4c9a-be1f-b95f43f0eb8e None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:54.572685 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac7ba17-c8b8-4c9a-be1f-b95f43f0eb8e None admin] PATH_INFO: `/v3/users/ebe70be515e944fd8fac78a193a6e9a3` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:54.574324 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac7ba17-c8b8-4c9a-be1f-b95f43f0eb8e None admin] RBAC: Authorizing `identity:get_user(user_id=ebe70be515e944fd8fac78a193a6e9a3)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:54.575784 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac7ba17-c8b8-4c9a-be1f-b95f43f0eb8e None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:54.577430 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 570/1151] 10.222.0.66 () {56 vars in 1177 bytes} [Mon Jul 18 02:16:54 2022] GET /identity/v3/users/ebe70be515e944fd8fac78a193a6e9a3 => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:16:54.583247 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b615df9-d5f4-4b14-bb82-fa89bfe843e2 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:54.681752 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b615df9-d5f4-4b14-bb82-fa89bfe843e2 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:54.713352 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b61f10-f6eb-4eec-b62e-c30276fa126e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:54.714223 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b61f10-f6eb-4eec-b62e-c30276fa126e None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:54.714504 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b61f10-f6eb-4eec-b62e-c30276fa126e None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:54.714773 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b61f10-f6eb-4eec-b62e-c30276fa126e None admin] PATH_INFO: `/v3/projects/149fc3c6e00d499ab731067be3c4c77f` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:54.715926 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b61f10-f6eb-4eec-b62e-c30276fa126e None admin] RBAC: Authorizing `identity:get_project(project_id=149fc3c6e00d499ab731067be3c4c77f)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:54.717345 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b61f10-f6eb-4eec-b62e-c30276fa126e None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:54.719057 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 582/1152] 10.222.0.66 () {56 vars in 1186 bytes} [Mon Jul 18 02:16:54 2022] GET /identity/v3/projects/149fc3c6e00d499ab731067be3c4c77f => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 02:16:54.725715 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac7ba17-c8b8-4c9a-be1f-b95f43f0eb8e None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:54.823097 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac7ba17-c8b8-4c9a-be1f-b95f43f0eb8e None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:54.855587 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc47ae4-0e0e-480c-b43c-99c392e70876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:54.856469 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc47ae4-0e0e-480c-b43c-99c392e70876 None admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:54.856755 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc47ae4-0e0e-480c-b43c-99c392e70876 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:54.857033 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc47ae4-0e0e-480c-b43c-99c392e70876 None admin] PATH_INFO: `/v3/projects/149fc3c6e00d499ab731067be3c4c77f/users/ebe70be515e944fd8fac78a193a6e9a3/roles/d4aa3324f0c84887b41d0dc4acf7f30e` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:54.859010 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc47ae4-0e0e-480c-b43c-99c392e70876 None admin] RBAC: Authorizing `identity:create_grant(project_id=149fc3c6e00d499ab731067be3c4c77f, user_id=ebe70be515e944fd8fac78a193a6e9a3, role_id=d4aa3324f0c84887b41d0dc4acf7f30e)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:54.860447 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc47ae4-0e0e-480c-b43c-99c392e70876 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:54.938495 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 571/1153] 10.222.0.66 () {58 vars in 1439 bytes} [Mon Jul 18 02:16:54 2022] PUT /identity/v3/projects/149fc3c6e00d499ab731067be3c4c77f/users/ebe70be515e944fd8fac78a193a6e9a3/roles/d4aa3324f0c84887b41d0dc4acf7f30e => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:16:56.900782 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2f93ed-dcb5-4376-b63f-8b1605d0aaed None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:56.901362 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2f93ed-dcb5-4376-b63f-8b1605d0aaed None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:56.901362 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2f93ed-dcb5-4376-b63f-8b1605d0aaed None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:56.902642 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 583/1154] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:16:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:16:56.907909 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee59fea-5b34-4196-8a1b-b988a9bf961b None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:56.908333 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee59fea-5b34-4196-8a1b-b988a9bf961b None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:56.908458 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee59fea-5b34-4196-8a1b-b988a9bf961b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:56.926219 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-5ee59fea-5b34-4196-8a1b-b988a9bf961b None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:16:56.927397 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-5ee59fea-5b34-4196-8a1b-b988a9bf961b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:16:56.955005 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:56.993044 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 572/1155] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:16:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 02:16:57.016601 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2f93ed-dcb5-4376-b63f-8b1605d0aaed None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:57.020261 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:57.114634 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2f93ed-dcb5-4376-b63f-8b1605d0aaed None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:57.146563 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2355b970-b572-4716-8cba-0f556469af13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:57.147331 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2355b970-b572-4716-8cba-0f556469af13 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:57.147616 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2355b970-b572-4716-8cba-0f556469af13 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:57.147892 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2355b970-b572-4716-8cba-0f556469af13 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:57.158056 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2355b970-b572-4716-8cba-0f556469af13 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 02:16:57.166094 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-2355b970-b572-4716-8cba-0f556469af13 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 02:16:57.166094 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:16:57.166094 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:16:57.166094 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:16:57.166094 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:16:57.166094 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:16:57.166094 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:16:57.166094 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:16:57.166094 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:16:57.166094 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:16:57.166094 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:16:57.166094 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:16:57.166094 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 02:16:57.166094 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 02:16:57.166094 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 02:16:57.166094 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 02:16:57.167519 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 02:16:57.167519 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 02:16:57.167519 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 02:16:57.167519 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 02:16:57.167519 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:16:57.167519 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:16:57.167519 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:16:57.167519 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:16:57.167519 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:16:57.167519 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:16:57.167519 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:16:57.167519 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:16:57.167519 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:16:57.167519 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:16:57.167519 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:16:57.167519 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:16:57.168408 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:16:57.168408 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:16:57.168408 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:16:57.168408 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:16:57.168408 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 02:16:57.168408 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 02:16:57.168408 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 02:16:57.168408 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 02:16:57.168408 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 02:16:57.168408 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 02:16:57.168408 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 02:16:57.168408 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:16:57.168408 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 584/1156] 10.222.0.66 () {56 vars in 1096 bytes} [Mon Jul 18 02:16:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:16:57.173636 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee59fea-5b34-4196-8a1b-b988a9bf961b None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:57.268659 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee59fea-5b34-4196-8a1b-b988a9bf961b None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:57.302163 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1822676-6563-4645-8b07-234a2379c40d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:57.302813 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1822676-6563-4645-8b07-234a2379c40d None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:57.303087 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1822676-6563-4645-8b07-234a2379c40d None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:57.303357 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1822676-6563-4645-8b07-234a2379c40d None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:57.304087 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1822676-6563-4645-8b07-234a2379c40d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:16:57.304390 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1822676-6563-4645-8b07-234a2379c40d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:57.305750 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1822676-6563-4645-8b07-234a2379c40d None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:57.313525 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 573/1157] 10.222.0.66 () {56 vars in 1099 bytes} [Mon Jul 18 02:16:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 02:16:57.319290 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2355b970-b572-4716-8cba-0f556469af13 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:57.379445 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:57.416777 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2355b970-b572-4716-8cba-0f556469af13 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:57.449737 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19acca7a-adf5-4a6c-84f8-a3294d91242b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:57.450615 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19acca7a-adf5-4a6c-84f8-a3294d91242b None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:57.450902 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19acca7a-adf5-4a6c-84f8-a3294d91242b None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:57.451179 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19acca7a-adf5-4a6c-84f8-a3294d91242b None admin] PATH_INFO: `/v3/users/ebe70be515e944fd8fac78a193a6e9a3` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:57.452389 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19acca7a-adf5-4a6c-84f8-a3294d91242b None admin] RBAC: Authorizing `identity:get_user(user_id=ebe70be515e944fd8fac78a193a6e9a3)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:57.453819 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19acca7a-adf5-4a6c-84f8-a3294d91242b None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:57.455509 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 585/1158] 10.222.0.66 () {56 vars in 1177 bytes} [Mon Jul 18 02:16:57 2022] GET /identity/v3/users/ebe70be515e944fd8fac78a193a6e9a3 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:16:57.461822 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1822676-6563-4645-8b07-234a2379c40d None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:57.490620 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:57.561191 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1822676-6563-4645-8b07-234a2379c40d None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:57.594470 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0dffc2-dbd9-4221-996f-328cd28ddec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:57.595108 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0dffc2-dbd9-4221-996f-328cd28ddec9 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:57.595394 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0dffc2-dbd9-4221-996f-328cd28ddec9 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:57.595671 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0dffc2-dbd9-4221-996f-328cd28ddec9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:57.663137 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0dffc2-dbd9-4221-996f-328cd28ddec9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:57.664592 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0dffc2-dbd9-4221-996f-328cd28ddec9 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:57.690346 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:57.760722 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 574/1159] 10.222.0.66 () {58 vars in 1282 bytes} [Mon Jul 18 02:16:57 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 02:16:57.766685 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19acca7a-adf5-4a6c-84f8-a3294d91242b None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:57.864372 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19acca7a-adf5-4a6c-84f8-a3294d91242b None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:57.892101 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:57.898602 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7599b32-9150-450c-a706-b87a54221be3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:57.899451 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7599b32-9150-450c-a706-b87a54221be3 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:57.899737 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7599b32-9150-450c-a706-b87a54221be3 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:57.900012 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7599b32-9150-450c-a706-b87a54221be3 None admin] PATH_INFO: `/v3/projects/149fc3c6e00d499ab731067be3c4c77f` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:57.901182 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7599b32-9150-450c-a706-b87a54221be3 None admin] RBAC: Authorizing `identity:get_project(project_id=149fc3c6e00d499ab731067be3c4c77f)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:57.902648 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7599b32-9150-450c-a706-b87a54221be3 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:57.904348 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 586/1160] 10.222.0.66 () {56 vars in 1186 bytes} [Mon Jul 18 02:16:57 2022] GET /identity/v3/projects/149fc3c6e00d499ab731067be3c4c77f => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:16:57.910480 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0dffc2-dbd9-4221-996f-328cd28ddec9 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:16:58.007510 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0dffc2-dbd9-4221-996f-328cd28ddec9 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:16:58.032626 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:16:58.040349 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36e5233-dacb-4186-98ed-c88596a82767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:16:58.040989 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36e5233-dacb-4186-98ed-c88596a82767 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:16:58.041276 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36e5233-dacb-4186-98ed-c88596a82767 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:16:58.043286 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36e5233-dacb-4186-98ed-c88596a82767 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:16:58.043286 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36e5233-dacb-4186-98ed-c88596a82767 None admin] RBAC: Adding query filter params (role.id=d4aa3324f0c84887b41d0dc4acf7f30e, scope.project.id=149fc3c6e00d499ab731067be3c4c77f, user.id=ebe70be515e944fd8fac78a193a6e9a3) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:16:58.043286 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36e5233-dacb-4186-98ed-c88596a82767 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:16:58.044503 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36e5233-dacb-4186-98ed-c88596a82767 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:16:58.050293 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 575/1161] 10.222.0.66 () {56 vars in 1374 bytes} [Mon Jul 18 02:16:57 2022] GET /identity/v3/role_assignments?user.id=ebe70be515e944fd8fac78a193a6e9a3&scope.project.id=149fc3c6e00d499ab731067be3c4c77f&role.id=d4aa3324f0c84887b41d0dc4acf7f30e => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:17:00.083476 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395a06a7-f98f-4dbd-ba54-215211447564 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:17:00.084014 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395a06a7-f98f-4dbd-ba54-215211447564 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:17:00.084014 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395a06a7-f98f-4dbd-ba54-215211447564 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:17:00.085433 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 587/1162] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:17:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:17:00.090554 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d927a0a-0c81-49c8-8791-3691628d15b6 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:17:00.090928 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d927a0a-0c81-49c8-8791-3691628d15b6 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:17:00.091116 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d927a0a-0c81-49c8-8791-3691628d15b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:17:00.108089 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-4d927a0a-0c81-49c8-8791-3691628d15b6 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:17:00.109263 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-4d927a0a-0c81-49c8-8791-3691628d15b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:17:00.174432 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 576/1163] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:17:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 02:17:00.196477 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395a06a7-f98f-4dbd-ba54-215211447564 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:17:00.295506 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395a06a7-f98f-4dbd-ba54-215211447564 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:17:00.329267 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1333bcb-ae07-4ad6-8544-e63469b90398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:17:00.329892 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1333bcb-ae07-4ad6-8544-e63469b90398 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:17:00.330205 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1333bcb-ae07-4ad6-8544-e63469b90398 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:17:00.330483 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1333bcb-ae07-4ad6-8544-e63469b90398 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:17:00.392868 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:17:00.396688 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1333bcb-ae07-4ad6-8544-e63469b90398 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:17:00.398241 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1333bcb-ae07-4ad6-8544-e63469b90398 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:17:00.493205 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 588/1164] 10.222.0.66 () {58 vars in 1282 bytes} [Mon Jul 18 02:17:00 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 02:17:00.499306 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d927a0a-0c81-49c8-8791-3691628d15b6 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:17:00.594976 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d927a0a-0c81-49c8-8791-3691628d15b6 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:17:00.617680 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:17:00.629323 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb157cf8-1f91-4d4f-8bbf-814ed8451411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:17:00.631170 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb157cf8-1f91-4d4f-8bbf-814ed8451411 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:17:00.631170 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb157cf8-1f91-4d4f-8bbf-814ed8451411 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:17:00.631170 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb157cf8-1f91-4d4f-8bbf-814ed8451411 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:17:00.637443 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb157cf8-1f91-4d4f-8bbf-814ed8451411 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:17:00.638924 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb157cf8-1f91-4d4f-8bbf-814ed8451411 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:17:00.654952 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-cb157cf8-1f91-4d4f-8bbf-814ed8451411 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 18 02:17:00.654952 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:17:00.654952 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 18 02:17:00.654952 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 18 02:17:00.654952 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 02:17:00.654952 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 02:17:00.654952 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 02:17:00.654952 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 02:17:00.654952 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 18 02:17:00.654952 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:17:00.654952 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 02:17:00.654952 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:17:00.654952 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:17:00.654952 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:17:00.654952 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:17:00.654952 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:17:00.654952 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:17:00.656211 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:17:00.656211 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:17:00.656211 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:17:00.656211 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:17:00.656211 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:17:00.656211 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:17:00.656211 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 02:17:00.656211 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 02:17:00.656211 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 02:17:00.656211 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 02:17:00.656211 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:17:00.656211 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:17:00.656211 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:17:00.656211 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:17:00.656211 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:17:00.656211 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:17:00.657262 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:17:00.657262 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:17:00.657262 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:17:00.657262 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:17:00.657262 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:17:00.657262 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:17:00.657262 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:17:00.657262 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:17:00.657262 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:17:00.657262 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:17:00.657262 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 18 02:17:00.657262 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 02:17:00.657262 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 18 02:17:00.657262 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:17:00.657262 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 577/1165] 10.222.0.66 () {56 vars in 1117 bytes} [Mon Jul 18 02:17:00 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:17:00.662511 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1333bcb-ae07-4ad6-8544-e63469b90398 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:17:00.754177 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:17:00.757856 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1333bcb-ae07-4ad6-8544-e63469b90398 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:17:00.790366 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e084fd67-f5e4-4f34-8c8f-85dbe7f40a3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:17:00.791012 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e084fd67-f5e4-4f34-8c8f-85dbe7f40a3b None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:17:00.791288 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e084fd67-f5e4-4f34-8c8f-85dbe7f40a3b None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:17:00.791559 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e084fd67-f5e4-4f34-8c8f-85dbe7f40a3b None admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:17:00.792528 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e084fd67-f5e4-4f34-8c8f-85dbe7f40a3b None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:17:00.792825 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e084fd67-f5e4-4f34-8c8f-85dbe7f40a3b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:17:00.794200 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e084fd67-f5e4-4f34-8c8f-85dbe7f40a3b None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:17:00.801306 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 589/1166] 10.222.0.66 () {56 vars in 1115 bytes} [Mon Jul 18 02:17:00 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:17:02.794427 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0173fb4d-5077-40ed-aa0d-2dd3bad1c08f None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:17:02.794958 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0173fb4d-5077-40ed-aa0d-2dd3bad1c08f None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:17:02.795078 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0173fb4d-5077-40ed-aa0d-2dd3bad1c08f None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:17:02.796373 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 578/1167] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:17:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:17:02.802118 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a537d97-a642-4ffe-89da-c90d7af0e3cb None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:17:02.802533 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a537d97-a642-4ffe-89da-c90d7af0e3cb None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:17:02.802674 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a537d97-a642-4ffe-89da-c90d7af0e3cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:17:02.819658 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-5a537d97-a642-4ffe-89da-c90d7af0e3cb None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:17:02.820830 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-5a537d97-a642-4ffe-89da-c90d7af0e3cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:17:02.884094 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 590/1168] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:17:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 02:17:02.906064 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0173fb4d-5077-40ed-aa0d-2dd3bad1c08f None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:17:03.007148 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0173fb4d-5077-40ed-aa0d-2dd3bad1c08f None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:17:03.039739 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672cf622-4e9b-4fb3-8aea-5d25f360890e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:17:03.040398 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672cf622-4e9b-4fb3-8aea-5d25f360890e None admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:17:03.040707 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672cf622-4e9b-4fb3-8aea-5d25f360890e None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:17:03.040980 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672cf622-4e9b-4fb3-8aea-5d25f360890e None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:17:03.041714 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672cf622-4e9b-4fb3-8aea-5d25f360890e None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:17:03.043370 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672cf622-4e9b-4fb3-8aea-5d25f360890e None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:17:03.122232 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 579/1169] 10.222.0.66 () {60 vars in 1138 bytes} [Mon Jul 18 02:17:02 2022] POST /identity/v3/domains => generated 267 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:17:05.142665 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2dd1f4-bc48-4121-9d6b-61e15600c199 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:17:05.143205 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2dd1f4-bc48-4121-9d6b-61e15600c199 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:17:05.143205 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2dd1f4-bc48-4121-9d6b-61e15600c199 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:17:05.144665 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 591/1170] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:17:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:17:05.151308 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b1f219-325b-4870-9155-1fca3d3dd530 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:17:05.151974 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b1f219-325b-4870-9155-1fca3d3dd530 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:17:05.151974 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b1f219-325b-4870-9155-1fca3d3dd530 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:17:05.168513 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-41b1f219-325b-4870-9155-1fca3d3dd530 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:17:05.169679 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-41b1f219-325b-4870-9155-1fca3d3dd530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:17:05.203152 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:17:05.233712 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 580/1171] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:17:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 02:17:05.256224 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2dd1f4-bc48-4121-9d6b-61e15600c199 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:17:05.345008 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:17:05.354628 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2dd1f4-bc48-4121-9d6b-61e15600c199 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:17:05.387402 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d941ba-593c-4a43-ba91-e1f3f43c5421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:17:05.388244 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d941ba-593c-4a43-ba91-e1f3f43c5421 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:17:05.388528 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d941ba-593c-4a43-ba91-e1f3f43c5421 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:17:05.388802 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d941ba-593c-4a43-ba91-e1f3f43c5421 None admin] PATH_INFO: `/v3/domains/8f069c7bef994579b758d20efc3af8a8` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:17:05.399772 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d941ba-593c-4a43-ba91-e1f3f43c5421 None admin] RBAC: Authorizing `identity:get_domain(domain_id=8f069c7bef994579b758d20efc3af8a8)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:17:05.401152 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d941ba-593c-4a43-ba91-e1f3f43c5421 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:17:05.402809 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 592/1172] 10.222.0.66 () {56 vars in 1183 bytes} [Mon Jul 18 02:17:05 2022] GET /identity/v3/domains/8f069c7bef994579b758d20efc3af8a8 => generated 267 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:17:05.408757 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b1f219-325b-4870-9155-1fca3d3dd530 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:17:05.507555 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b1f219-325b-4870-9155-1fca3d3dd530 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:17:05.539875 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312413ec-bc04-49e1-aec9-30947f78ccd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:17:05.540523 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312413ec-bc04-49e1-aec9-30947f78ccd4 None admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:17:05.540809 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312413ec-bc04-49e1-aec9-30947f78ccd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:17:05.541087 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312413ec-bc04-49e1-aec9-30947f78ccd4 None admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:17:05.542068 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312413ec-bc04-49e1-aec9-30947f78ccd4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:17:05.543534 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312413ec-bc04-49e1-aec9-30947f78ccd4 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:17:05.668621 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 581/1173] 10.222.0.66 () {60 vars in 1141 bytes} [Mon Jul 18 02:17:05 2022] POST /identity/v3/projects => generated 361 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 02:17:07.715132 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c378856-bdc0-4436-9f11-61ddd8454607 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:17:07.715681 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c378856-bdc0-4436-9f11-61ddd8454607 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:17:07.715681 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c378856-bdc0-4436-9f11-61ddd8454607 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:17:07.717107 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 593/1174] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:17:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:17:07.722835 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc8b575-6059-4cd5-a935-414fa5dbbf14 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:17:07.723258 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc8b575-6059-4cd5-a935-414fa5dbbf14 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:17:07.723390 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc8b575-6059-4cd5-a935-414fa5dbbf14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:17:07.740297 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-2cc8b575-6059-4cd5-a935-414fa5dbbf14 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:17:07.741457 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-2cc8b575-6059-4cd5-a935-414fa5dbbf14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:17:07.758866 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:17:07.807932 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 582/1175] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:17:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 02:17:07.832760 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c378856-bdc0-4436-9f11-61ddd8454607 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:17:07.932721 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c378856-bdc0-4436-9f11-61ddd8454607 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:17:07.968085 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d25983-953c-468a-9d0f-9ebf7a06f23a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:17:07.968935 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d25983-953c-468a-9d0f-9ebf7a06f23a None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:17:07.969221 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d25983-953c-468a-9d0f-9ebf7a06f23a None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:17:07.969500 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d25983-953c-468a-9d0f-9ebf7a06f23a None admin] PATH_INFO: `/v3/domains/8f069c7bef994579b758d20efc3af8a8` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:17:07.970690 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d25983-953c-468a-9d0f-9ebf7a06f23a None admin] RBAC: Authorizing `identity:get_domain(domain_id=8f069c7bef994579b758d20efc3af8a8)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:17:07.972169 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d25983-953c-468a-9d0f-9ebf7a06f23a None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:17:07.973783 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 594/1176] 10.222.0.66 () {56 vars in 1183 bytes} [Mon Jul 18 02:17:07 2022] GET /identity/v3/domains/8f069c7bef994579b758d20efc3af8a8 => generated 267 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:17:07.979627 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc8b575-6059-4cd5-a935-414fa5dbbf14 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:17:08.076633 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc8b575-6059-4cd5-a935-414fa5dbbf14 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:17:08.109648 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64919ba-c7e7-47ee-b6ba-e2d01325665c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:17:08.110337 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64919ba-c7e7-47ee-b6ba-e2d01325665c None admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:17:08.110625 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64919ba-c7e7-47ee-b6ba-e2d01325665c None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:17:08.110909 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64919ba-c7e7-47ee-b6ba-e2d01325665c None admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:17:08.111609 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64919ba-c7e7-47ee-b6ba-e2d01325665c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:17:08.113101 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64919ba-c7e7-47ee-b6ba-e2d01325665c None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:17:08.198279 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 583/1177] 10.222.0.66 () {60 vars in 1132 bytes} [Mon Jul 18 02:17:07 2022] POST /identity/v3/users => generated 311 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:17:10.240787 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e9dc77-04b6-4ea8-aa51-2393f3ed3896 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:17:10.241331 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e9dc77-04b6-4ea8-aa51-2393f3ed3896 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:17:10.241331 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e9dc77-04b6-4ea8-aa51-2393f3ed3896 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:17:10.242743 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 595/1178] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:17:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:17:10.248129 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5166bf2d-cda0-4f8e-8e1e-836f0ba9dbe4 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:17:10.248561 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5166bf2d-cda0-4f8e-8e1e-836f0ba9dbe4 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:17:10.248703 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5166bf2d-cda0-4f8e-8e1e-836f0ba9dbe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:17:10.265748 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-5166bf2d-cda0-4f8e-8e1e-836f0ba9dbe4 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:17:10.266959 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-5166bf2d-cda0-4f8e-8e1e-836f0ba9dbe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:17:10.269848 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:17:10.332431 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 584/1179] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:17:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 02:17:10.356620 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e9dc77-04b6-4ea8-aa51-2393f3ed3896 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:17:10.379960 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:17:10.457702 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e9dc77-04b6-4ea8-aa51-2393f3ed3896 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:17:10.490256 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01263b25-de73-4a87-9a8a-f49d2a5f1c49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:17:10.491081 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01263b25-de73-4a87-9a8a-f49d2a5f1c49 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:17:10.491311 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01263b25-de73-4a87-9a8a-f49d2a5f1c49 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:17:10.491580 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01263b25-de73-4a87-9a8a-f49d2a5f1c49 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:17:10.501624 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01263b25-de73-4a87-9a8a-f49d2a5f1c49 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 02:17:10.509320 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-01263b25-de73-4a87-9a8a-f49d2a5f1c49 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 02:17:10.509320 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:17:10.509320 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:17:10.509320 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:17:10.509320 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:17:10.509320 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:17:10.509320 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:17:10.509320 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:17:10.509320 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:17:10.509320 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:17:10.509320 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:17:10.509320 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:17:10.509320 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 02:17:10.509320 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 02:17:10.509320 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 02:17:10.509320 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 02:17:10.510583 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 02:17:10.510583 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 02:17:10.510583 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 02:17:10.510583 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 02:17:10.510583 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:17:10.510583 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:17:10.510583 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:17:10.510583 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:17:10.510583 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:17:10.510583 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:17:10.510583 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:17:10.510583 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:17:10.510583 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:17:10.510583 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:17:10.510583 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:17:10.510583 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:17:10.512208 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:17:10.512208 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:17:10.512208 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:17:10.512208 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:17:10.512208 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 02:17:10.512208 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 02:17:10.512208 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 02:17:10.512208 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 02:17:10.512208 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 02:17:10.512208 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 02:17:10.512208 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 02:17:10.512208 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:17:10.512208 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 596/1180] 10.222.0.66 () {56 vars in 1096 bytes} [Mon Jul 18 02:17:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 02:17:10.517185 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5166bf2d-cda0-4f8e-8e1e-836f0ba9dbe4 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:17:10.615229 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5166bf2d-cda0-4f8e-8e1e-836f0ba9dbe4 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:17:10.647603 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4003ab-74ff-4f1b-894a-ab7ffd9ae3f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:17:10.648263 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4003ab-74ff-4f1b-894a-ab7ffd9ae3f8 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:17:10.648545 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4003ab-74ff-4f1b-894a-ab7ffd9ae3f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:17:10.648816 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4003ab-74ff-4f1b-894a-ab7ffd9ae3f8 None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:17:10.649556 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4003ab-74ff-4f1b-894a-ab7ffd9ae3f8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:17:10.649857 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4003ab-74ff-4f1b-894a-ab7ffd9ae3f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:17:10.651259 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4003ab-74ff-4f1b-894a-ab7ffd9ae3f8 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:17:10.659656 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 585/1181] 10.222.0.66 () {56 vars in 1099 bytes} [Mon Jul 18 02:17:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:17:10.665569 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01263b25-de73-4a87-9a8a-f49d2a5f1c49 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:17:10.745257 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:17:10.763952 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01263b25-de73-4a87-9a8a-f49d2a5f1c49 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:17:10.798341 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa954caa-9851-4495-abd4-3783f28f23c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:17:10.799461 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa954caa-9851-4495-abd4-3783f28f23c7 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:17:10.799756 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa954caa-9851-4495-abd4-3783f28f23c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:17:10.800030 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa954caa-9851-4495-abd4-3783f28f23c7 None admin] PATH_INFO: `/v3/users/a245cfaeeaff4dcd81eec06d14ec7450` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:17:10.812510 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa954caa-9851-4495-abd4-3783f28f23c7 None admin] RBAC: Authorizing `identity:get_user(user_id=a245cfaeeaff4dcd81eec06d14ec7450)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:17:10.813967 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa954caa-9851-4495-abd4-3783f28f23c7 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:17:10.815750 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 597/1182] 10.222.0.66 () {56 vars in 1177 bytes} [Mon Jul 18 02:17:10 2022] GET /identity/v3/users/a245cfaeeaff4dcd81eec06d14ec7450 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:17:10.821474 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4003ab-74ff-4f1b-894a-ab7ffd9ae3f8 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:17:10.824316 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:17:10.919089 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4003ab-74ff-4f1b-894a-ab7ffd9ae3f8 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:17:10.950565 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea1d3cb-3cf0-41fa-bb1c-8b2f0b011799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:17:10.951201 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea1d3cb-3cf0-41fa-bb1c-8b2f0b011799 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:17:10.951489 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea1d3cb-3cf0-41fa-bb1c-8b2f0b011799 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:17:10.951764 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea1d3cb-3cf0-41fa-bb1c-8b2f0b011799 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:17:11.016080 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea1d3cb-3cf0-41fa-bb1c-8b2f0b011799 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:17:11.017449 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea1d3cb-3cf0-41fa-bb1c-8b2f0b011799 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:17:11.019831 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:17:11.114078 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 586/1183] 10.222.0.66 () {58 vars in 1282 bytes} [Mon Jul 18 02:17:10 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 02:17:11.120059 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa954caa-9851-4495-abd4-3783f28f23c7 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:17:11.217414 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa954caa-9851-4495-abd4-3783f28f23c7 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:17:11.249960 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b525b464-c627-4a2e-ac1e-69bbdbe97653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:17:11.250828 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b525b464-c627-4a2e-ac1e-69bbdbe97653 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:17:11.251113 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b525b464-c627-4a2e-ac1e-69bbdbe97653 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:17:11.251392 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b525b464-c627-4a2e-ac1e-69bbdbe97653 None admin] PATH_INFO: `/v3/projects/47ffa262348742f3983c8f81d25d7720` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:17:11.253067 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b525b464-c627-4a2e-ac1e-69bbdbe97653 None admin] RBAC: Authorizing `identity:get_project(project_id=47ffa262348742f3983c8f81d25d7720)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:17:11.254537 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b525b464-c627-4a2e-ac1e-69bbdbe97653 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:17:11.256244 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 598/1184] 10.222.0.66 () {56 vars in 1186 bytes} [Mon Jul 18 02:17:11 2022] GET /identity/v3/projects/47ffa262348742f3983c8f81d25d7720 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:17:11.262298 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea1d3cb-3cf0-41fa-bb1c-8b2f0b011799 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:17:11.357625 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea1d3cb-3cf0-41fa-bb1c-8b2f0b011799 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:17:11.359072 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:17:11.391726 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369ef1ab-704d-4671-b940-16082f6f2eb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:17:11.393409 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369ef1ab-704d-4671-b940-16082f6f2eb2 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:17:11.394146 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369ef1ab-704d-4671-b940-16082f6f2eb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:17:11.394591 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369ef1ab-704d-4671-b940-16082f6f2eb2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:17:11.395645 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369ef1ab-704d-4671-b940-16082f6f2eb2 None admin] RBAC: Adding query filter params (role.id=d4aa3324f0c84887b41d0dc4acf7f30e, scope.project.id=47ffa262348742f3983c8f81d25d7720, user.id=a245cfaeeaff4dcd81eec06d14ec7450) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:17:11.396272 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369ef1ab-704d-4671-b940-16082f6f2eb2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:17:11.397719 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369ef1ab-704d-4671-b940-16082f6f2eb2 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:17:11.403784 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 587/1185] 10.222.0.66 () {56 vars in 1374 bytes} [Mon Jul 18 02:17:11 2022] GET /identity/v3/role_assignments?user.id=a245cfaeeaff4dcd81eec06d14ec7450&scope.project.id=47ffa262348742f3983c8f81d25d7720&role.id=d4aa3324f0c84887b41d0dc4acf7f30e => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:17:13.381812 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c46c5b5-8b3b-4582-b4c9-4b4f3a4537bf None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:17:13.382439 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c46c5b5-8b3b-4582-b4c9-4b4f3a4537bf None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:17:13.382439 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c46c5b5-8b3b-4582-b4c9-4b4f3a4537bf None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:17:13.383752 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 599/1186] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:17:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:17:13.388940 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95737b1-59bf-4430-9ba1-746145156891 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:17:13.389384 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95737b1-59bf-4430-9ba1-746145156891 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:17:13.389486 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95737b1-59bf-4430-9ba1-746145156891 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:17:13.406704 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-b95737b1-59bf-4430-9ba1-746145156891 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:17:13.407961 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-b95737b1-59bf-4430-9ba1-746145156891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:17:13.474563 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 588/1187] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:17:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 02:17:13.497632 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c46c5b5-8b3b-4582-b4c9-4b4f3a4537bf None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:17:13.506910 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:17:13.598931 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c46c5b5-8b3b-4582-b4c9-4b4f3a4537bf None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:17:13.632800 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ec0317-04c6-4802-9013-5783a1498aab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:17:13.633526 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ec0317-04c6-4802-9013-5783a1498aab None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:17:13.633813 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ec0317-04c6-4802-9013-5783a1498aab None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:17:13.634115 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ec0317-04c6-4802-9013-5783a1498aab None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:17:13.644675 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ec0317-04c6-4802-9013-5783a1498aab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 02:17:13.652306 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-73ec0317-04c6-4802-9013-5783a1498aab None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 02:17:13.652306 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:17:13.652306 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:17:13.652306 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:17:13.652306 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:17:13.652306 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:17:13.652306 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:17:13.652306 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:17:13.652306 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:17:13.652306 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:17:13.652306 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:17:13.652306 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:17:13.652306 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 02:17:13.652306 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 02:17:13.652306 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 02:17:13.652306 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 02:17:13.653333 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 02:17:13.653333 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 02:17:13.653333 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 02:17:13.653333 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 02:17:13.653333 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:17:13.653333 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:17:13.653333 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:17:13.653333 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:17:13.653333 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:17:13.653333 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:17:13.653333 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:17:13.653333 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:17:13.653333 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:17:13.653333 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:17:13.653333 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:17:13.653333 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:17:13.654618 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:17:13.654618 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:17:13.654618 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:17:13.654618 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:17:13.654618 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 02:17:13.654618 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 02:17:13.654618 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 02:17:13.654618 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 02:17:13.654618 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 02:17:13.654618 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 02:17:13.654618 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 02:17:13.654618 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:17:13.654618 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 600/1188] 10.222.0.66 () {56 vars in 1096 bytes} [Mon Jul 18 02:17:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:17:13.659707 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95737b1-59bf-4430-9ba1-746145156891 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:17:13.751394 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:17:13.757501 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95737b1-59bf-4430-9ba1-746145156891 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:17:13.790260 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f6b093-b1b4-40f6-ad7a-a5572fb27ac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:17:13.790920 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f6b093-b1b4-40f6-ad7a-a5572fb27ac9 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:17:13.791455 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f6b093-b1b4-40f6-ad7a-a5572fb27ac9 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:17:13.791730 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f6b093-b1b4-40f6-ad7a-a5572fb27ac9 None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:17:13.792463 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f6b093-b1b4-40f6-ad7a-a5572fb27ac9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:17:13.792797 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f6b093-b1b4-40f6-ad7a-a5572fb27ac9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:17:13.794212 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f6b093-b1b4-40f6-ad7a-a5572fb27ac9 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:17:13.802628 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 589/1189] 10.222.0.66 () {56 vars in 1099 bytes} [Mon Jul 18 02:17:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 02:17:13.808419 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ec0317-04c6-4802-9013-5783a1498aab None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:17:13.872196 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:17:13.905471 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ec0317-04c6-4802-9013-5783a1498aab None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:17:13.938224 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0cd41e-9899-401d-a4d3-2b5ea9f803ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:17:13.939070 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0cd41e-9899-401d-a4d3-2b5ea9f803ae None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:17:13.939496 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0cd41e-9899-401d-a4d3-2b5ea9f803ae None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:17:13.939777 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0cd41e-9899-401d-a4d3-2b5ea9f803ae None admin] PATH_INFO: `/v3/users/a245cfaeeaff4dcd81eec06d14ec7450` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:17:13.940943 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0cd41e-9899-401d-a4d3-2b5ea9f803ae None admin] RBAC: Authorizing `identity:get_user(user_id=a245cfaeeaff4dcd81eec06d14ec7450)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:17:13.942413 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0cd41e-9899-401d-a4d3-2b5ea9f803ae None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:17:13.944030 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 601/1190] 10.222.0.66 () {56 vars in 1177 bytes} [Mon Jul 18 02:17:13 2022] GET /identity/v3/users/a245cfaeeaff4dcd81eec06d14ec7450 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:17:13.949770 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f6b093-b1b4-40f6-ad7a-a5572fb27ac9 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:17:14.047621 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f6b093-b1b4-40f6-ad7a-a5572fb27ac9 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:17:14.080380 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46b4e82-29cf-4b62-9105-88dc0a6e91c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:17:14.081226 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46b4e82-29cf-4b62-9105-88dc0a6e91c6 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:17:14.081507 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46b4e82-29cf-4b62-9105-88dc0a6e91c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:17:14.081804 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46b4e82-29cf-4b62-9105-88dc0a6e91c6 None admin] PATH_INFO: `/v3/projects/47ffa262348742f3983c8f81d25d7720` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:17:14.083049 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46b4e82-29cf-4b62-9105-88dc0a6e91c6 None admin] RBAC: Authorizing `identity:get_project(project_id=47ffa262348742f3983c8f81d25d7720)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:17:14.084485 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46b4e82-29cf-4b62-9105-88dc0a6e91c6 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:17:14.086387 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 590/1191] 10.222.0.66 () {56 vars in 1186 bytes} [Mon Jul 18 02:17:13 2022] GET /identity/v3/projects/47ffa262348742f3983c8f81d25d7720 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:17:14.092182 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0cd41e-9899-401d-a4d3-2b5ea9f803ae None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:17:14.191213 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0cd41e-9899-401d-a4d3-2b5ea9f803ae None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:17:14.223458 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd93f8dc-388f-4435-aedc-e8b88ff20dd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:17:14.224348 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd93f8dc-388f-4435-aedc-e8b88ff20dd5 None admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:17:14.224628 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd93f8dc-388f-4435-aedc-e8b88ff20dd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:17:14.224900 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd93f8dc-388f-4435-aedc-e8b88ff20dd5 None admin] PATH_INFO: `/v3/projects/47ffa262348742f3983c8f81d25d7720/users/a245cfaeeaff4dcd81eec06d14ec7450/roles/d4aa3324f0c84887b41d0dc4acf7f30e` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:17:14.226808 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd93f8dc-388f-4435-aedc-e8b88ff20dd5 None admin] RBAC: Authorizing `identity:create_grant(project_id=47ffa262348742f3983c8f81d25d7720, user_id=a245cfaeeaff4dcd81eec06d14ec7450, role_id=d4aa3324f0c84887b41d0dc4acf7f30e)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:17:14.228219 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd93f8dc-388f-4435-aedc-e8b88ff20dd5 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:17:14.230392 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:17:14.302695 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 602/1192] 10.222.0.66 () {58 vars in 1439 bytes} [Mon Jul 18 02:17:14 2022] PUT /identity/v3/projects/47ffa262348742f3983c8f81d25d7720/users/a245cfaeeaff4dcd81eec06d14ec7450/roles/d4aa3324f0c84887b41d0dc4acf7f30e => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:17:16.285246 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbae5981-6c75-417d-97f0-6f18efa1c3ca None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:17:16.285825 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbae5981-6c75-417d-97f0-6f18efa1c3ca None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:17:16.285825 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbae5981-6c75-417d-97f0-6f18efa1c3ca None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:17:16.287236 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 591/1193] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:17:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:17:16.292848 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5dce48-2872-4c99-8b72-8bb81a8788fe None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:17:16.293259 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5dce48-2872-4c99-8b72-8bb81a8788fe None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:17:16.293401 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5dce48-2872-4c99-8b72-8bb81a8788fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:17:16.310003 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-be5dce48-2872-4c99-8b72-8bb81a8788fe None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:17:16.311231 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-be5dce48-2872-4c99-8b72-8bb81a8788fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:17:16.374540 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 603/1194] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:17:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 02:17:16.398185 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbae5981-6c75-417d-97f0-6f18efa1c3ca None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:17:16.426715 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:17:16.499774 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbae5981-6c75-417d-97f0-6f18efa1c3ca None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:17:16.532127 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8fb93c-e85f-4ef2-a09a-f665e07f6fab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:17:16.533717 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8fb93c-e85f-4ef2-a09a-f665e07f6fab None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:17:16.534002 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8fb93c-e85f-4ef2-a09a-f665e07f6fab None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:17:16.534305 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8fb93c-e85f-4ef2-a09a-f665e07f6fab None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:17:16.544413 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8fb93c-e85f-4ef2-a09a-f665e07f6fab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 02:17:16.552783 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-7b8fb93c-e85f-4ef2-a09a-f665e07f6fab None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 02:17:16.552783 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:17:16.552783 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:17:16.552783 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:17:16.552783 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:17:16.552783 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:17:16.552783 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:17:16.552783 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:17:16.552783 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:17:16.552783 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:17:16.552783 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:17:16.552783 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:17:16.552783 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 02:17:16.552783 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 02:17:16.552783 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 02:17:16.552783 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 02:17:16.553882 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 02:17:16.553882 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 02:17:16.553882 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 02:17:16.553882 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 02:17:16.553882 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:17:16.553882 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:17:16.553882 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:17:16.553882 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:17:16.553882 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:17:16.553882 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:17:16.553882 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:17:16.553882 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:17:16.553882 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:17:16.553882 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:17:16.553882 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:17:16.553882 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:17:16.555101 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:17:16.555101 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:17:16.555101 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:17:16.555101 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:17:16.555101 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 02:17:16.555101 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 02:17:16.555101 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 02:17:16.555101 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 02:17:16.555101 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 02:17:16.555101 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 02:17:16.555101 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 02:17:16.555101 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:17:16.555101 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 592/1195] 10.222.0.66 () {56 vars in 1096 bytes} [Mon Jul 18 02:17:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:17:16.559974 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5dce48-2872-4c99-8b72-8bb81a8788fe None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:17:16.660312 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5dce48-2872-4c99-8b72-8bb81a8788fe None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:17:16.683689 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:17:16.694110 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606ddf1b-4a60-4760-89b3-55b7cf8bc111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:17:16.695922 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606ddf1b-4a60-4760-89b3-55b7cf8bc111 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:17:16.695922 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606ddf1b-4a60-4760-89b3-55b7cf8bc111 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:17:16.695922 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606ddf1b-4a60-4760-89b3-55b7cf8bc111 None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:17:16.696178 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606ddf1b-4a60-4760-89b3-55b7cf8bc111 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:17:16.696391 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606ddf1b-4a60-4760-89b3-55b7cf8bc111 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:17:16.697756 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606ddf1b-4a60-4760-89b3-55b7cf8bc111 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:17:16.706106 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 604/1196] 10.222.0.66 () {56 vars in 1099 bytes} [Mon Jul 18 02:17:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:17:16.712089 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8fb93c-e85f-4ef2-a09a-f665e07f6fab None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:17:16.796352 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:17:16.810892 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8fb93c-e85f-4ef2-a09a-f665e07f6fab None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:17:16.844698 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5cf4c3-1599-44c4-8270-0eda519741ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:17:16.846388 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5cf4c3-1599-44c4-8270-0eda519741ab None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:17:16.846668 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5cf4c3-1599-44c4-8270-0eda519741ab None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:17:16.846945 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5cf4c3-1599-44c4-8270-0eda519741ab None admin] PATH_INFO: `/v3/users/a245cfaeeaff4dcd81eec06d14ec7450` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:17:16.848126 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5cf4c3-1599-44c4-8270-0eda519741ab None admin] RBAC: Authorizing `identity:get_user(user_id=a245cfaeeaff4dcd81eec06d14ec7450)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:17:16.849908 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5cf4c3-1599-44c4-8270-0eda519741ab None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:17:16.851580 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 593/1197] 10.222.0.66 () {56 vars in 1177 bytes} [Mon Jul 18 02:17:16 2022] GET /identity/v3/users/a245cfaeeaff4dcd81eec06d14ec7450 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:17:16.857316 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606ddf1b-4a60-4760-89b3-55b7cf8bc111 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:17:16.953470 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606ddf1b-4a60-4760-89b3-55b7cf8bc111 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:17:16.986607 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d157381-2625-4d18-944f-93c151bdf415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:17:16.987243 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d157381-2625-4d18-944f-93c151bdf415 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:17:16.987530 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d157381-2625-4d18-944f-93c151bdf415 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:17:16.987804 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d157381-2625-4d18-944f-93c151bdf415 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:17:17.039766 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:17:17.055284 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d157381-2625-4d18-944f-93c151bdf415 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:17:17.055284 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d157381-2625-4d18-944f-93c151bdf415 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:17:17.149853 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 605/1198] 10.222.0.66 () {58 vars in 1282 bytes} [Mon Jul 18 02:17:16 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 02:17:17.155669 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5cf4c3-1599-44c4-8270-0eda519741ab None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:17:17.251948 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5cf4c3-1599-44c4-8270-0eda519741ab None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:17:17.284749 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e278e0ca-ef81-414d-bffb-5d323bafff5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:17:17.285589 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e278e0ca-ef81-414d-bffb-5d323bafff5c None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:17:17.286205 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e278e0ca-ef81-414d-bffb-5d323bafff5c None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:17:17.286205 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e278e0ca-ef81-414d-bffb-5d323bafff5c None admin] PATH_INFO: `/v3/projects/47ffa262348742f3983c8f81d25d7720` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:17:17.287295 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e278e0ca-ef81-414d-bffb-5d323bafff5c None admin] RBAC: Authorizing `identity:get_project(project_id=47ffa262348742f3983c8f81d25d7720)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:17:17.288729 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e278e0ca-ef81-414d-bffb-5d323bafff5c None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:17:17.290460 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 594/1199] 10.222.0.66 () {56 vars in 1186 bytes} [Mon Jul 18 02:17:17 2022] GET /identity/v3/projects/47ffa262348742f3983c8f81d25d7720 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:17:17.296804 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d157381-2625-4d18-944f-93c151bdf415 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:17:17.378569 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:17:17.393335 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d157381-2625-4d18-944f-93c151bdf415 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:17:17.426936 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02194d8-e9a2-4df2-9102-9a39d05624dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:17:17.427579 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02194d8-e9a2-4df2-9102-9a39d05624dd None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:17:17.427862 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02194d8-e9a2-4df2-9102-9a39d05624dd None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:17:17.428138 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02194d8-e9a2-4df2-9102-9a39d05624dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:17:17.428965 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02194d8-e9a2-4df2-9102-9a39d05624dd None admin] RBAC: Adding query filter params (role.id=d4aa3324f0c84887b41d0dc4acf7f30e, scope.project.id=47ffa262348742f3983c8f81d25d7720, user.id=a245cfaeeaff4dcd81eec06d14ec7450) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:17:17.429276 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02194d8-e9a2-4df2-9102-9a39d05624dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:17:17.430791 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02194d8-e9a2-4df2-9102-9a39d05624dd None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:17:17.436489 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 606/1200] 10.222.0.66 () {56 vars in 1374 bytes} [Mon Jul 18 02:17:17 2022] GET /identity/v3/role_assignments?user.id=a245cfaeeaff4dcd81eec06d14ec7450&scope.project.id=47ffa262348742f3983c8f81d25d7720&role.id=d4aa3324f0c84887b41d0dc4acf7f30e => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:17:27.280542 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=89 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:17:27.433838 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=25 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:18:31.264334 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f848b38-9b0d-46d7-9db0-386add09f699 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:18:31.264924 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f848b38-9b0d-46d7-9db0-386add09f699 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:18:31.264924 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f848b38-9b0d-46d7-9db0-386add09f699 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:18:31.266314 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 595/1201] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:18:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:18:31.271604 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188dea4e-fbf2-4b12-8878-ed362c3d4f06 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:18:31.272025 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188dea4e-fbf2-4b12-8878-ed362c3d4f06 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:18:31.272265 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188dea4e-fbf2-4b12-8878-ed362c3d4f06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:18:31.289006 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-188dea4e-fbf2-4b12-8878-ed362c3d4f06 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:18:31.290895 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-188dea4e-fbf2-4b12-8878-ed362c3d4f06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:18:31.358833 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 607/1202] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:18:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 02:18:31.384153 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f848b38-9b0d-46d7-9db0-386add09f699 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:18:31.486366 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f848b38-9b0d-46d7-9db0-386add09f699 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:18:31.521134 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90881d81-42ea-4dae-b620-b0cceb9b5e81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:18:31.521850 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90881d81-42ea-4dae-b620-b0cceb9b5e81 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:18:31.522150 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90881d81-42ea-4dae-b620-b0cceb9b5e81 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:18:31.522422 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90881d81-42ea-4dae-b620-b0cceb9b5e81 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:18:31.532409 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90881d81-42ea-4dae-b620-b0cceb9b5e81 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:18:31.533860 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90881d81-42ea-4dae-b620-b0cceb9b5e81 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:18:31.551650 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-90881d81-42ea-4dae-b620-b0cceb9b5e81 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:18:31.551650 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:18:31.551650 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:18:31.551650 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:18:31.551650 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:18:31.551650 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:18:31.551650 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:18:31.551650 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:18:31.551650 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:18:31.551650 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:18:31.551650 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:18:31.551650 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:18:31.551650 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 02:18:31.551650 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 02:18:31.551650 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 02:18:31.552835 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 02:18:31.552835 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:18:31.552835 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:18:31.552835 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:18:31.552835 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:18:31.552835 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:18:31.552835 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:18:31.552835 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:18:31.552835 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:18:31.552835 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:18:31.552835 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:18:31.552835 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:18:31.552835 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:18:31.552835 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:18:31.552835 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:18:31.552835 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:18:31.552835 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:18:31.554263 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 02:18:31.554263 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 02:18:31.554263 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:18:31.554263 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:18:31.554263 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 596/1203] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 18 02:18:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:18:31.559630 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188dea4e-fbf2-4b12-8878-ed362c3d4f06 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:18:31.661106 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188dea4e-fbf2-4b12-8878-ed362c3d4f06 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:18:31.688445 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:18:31.694030 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637f7931-98c7-4a94-8dbe-97097f33eb3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:18:31.696774 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637f7931-98c7-4a94-8dbe-97097f33eb3d None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:18:31.696774 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637f7931-98c7-4a94-8dbe-97097f33eb3d None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:18:31.696774 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637f7931-98c7-4a94-8dbe-97097f33eb3d None admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:18:31.696774 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637f7931-98c7-4a94-8dbe-97097f33eb3d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:18:31.697289 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637f7931-98c7-4a94-8dbe-97097f33eb3d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:18:31.698436 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637f7931-98c7-4a94-8dbe-97097f33eb3d None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:18:31.709932 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 608/1204] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 18 02:18:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 02:18:31.715938 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90881d81-42ea-4dae-b620-b0cceb9b5e81 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:18:31.771085 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:18:31.819843 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90881d81-42ea-4dae-b620-b0cceb9b5e81 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:18:31.853499 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195cb070-55f7-4ac6-b33e-76ae84a080e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:18:31.854195 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195cb070-55f7-4ac6-b33e-76ae84a080e0 None admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:18:31.854484 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195cb070-55f7-4ac6-b33e-76ae84a080e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:18:31.854763 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195cb070-55f7-4ac6-b33e-76ae84a080e0 None admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:18:31.855468 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195cb070-55f7-4ac6-b33e-76ae84a080e0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:18:31.856916 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195cb070-55f7-4ac6-b33e-76ae84a080e0 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:18:31.961786 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 597/1205] 10.222.0.66 () {60 vars in 1132 bytes} [Mon Jul 18 02:18:31 2022] POST /identity/v3/users => generated 251 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:18:34.039613 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d462e1df-5cf6-4e0d-b731-46e72b5efbc5 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:18:34.040176 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d462e1df-5cf6-4e0d-b731-46e72b5efbc5 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:18:34.040176 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d462e1df-5cf6-4e0d-b731-46e72b5efbc5 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:18:34.041544 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 609/1206] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:18:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:18:34.046902 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1a40da-eed9-4709-a30b-b3361372df2e None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:18:34.047328 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1a40da-eed9-4709-a30b-b3361372df2e None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:18:34.047491 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1a40da-eed9-4709-a30b-b3361372df2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:18:34.064088 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-6e1a40da-eed9-4709-a30b-b3361372df2e None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:18:34.065265 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-6e1a40da-eed9-4709-a30b-b3361372df2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:18:34.132555 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 598/1207] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:18:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 02:18:34.157054 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d462e1df-5cf6-4e0d-b731-46e72b5efbc5 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:18:34.258696 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d462e1df-5cf6-4e0d-b731-46e72b5efbc5 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:18:34.294166 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac1057a-779a-4a84-a8d1-ffe61fcf050d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:18:34.294899 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac1057a-779a-4a84-a8d1-ffe61fcf050d None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:18:34.295184 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac1057a-779a-4a84-a8d1-ffe61fcf050d None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:18:34.295460 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac1057a-779a-4a84-a8d1-ffe61fcf050d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:18:34.305318 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac1057a-779a-4a84-a8d1-ffe61fcf050d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 02:18:34.313801 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-7ac1057a-779a-4a84-a8d1-ffe61fcf050d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 02:18:34.313801 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:18:34.313801 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:18:34.313801 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:18:34.313801 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:18:34.313801 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:18:34.313801 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:18:34.313801 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:18:34.313801 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:18:34.313801 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:18:34.313801 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:18:34.313801 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:18:34.313801 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 02:18:34.313801 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 02:18:34.313801 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 02:18:34.313801 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 02:18:34.315036 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 02:18:34.315036 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 02:18:34.315036 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 02:18:34.315036 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 02:18:34.315036 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:18:34.315036 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:18:34.315036 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:18:34.315036 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:18:34.315036 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:18:34.315036 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:18:34.315036 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:18:34.315036 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:18:34.315036 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:18:34.315036 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:18:34.315036 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:18:34.315036 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:18:34.316179 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:18:34.316179 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:18:34.316179 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:18:34.316179 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:18:34.316179 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 02:18:34.316179 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 02:18:34.316179 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 02:18:34.316179 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 02:18:34.316179 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 02:18:34.316179 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 02:18:34.316179 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 02:18:34.316179 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:18:34.316179 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 610/1208] 10.222.0.66 () {56 vars in 1102 bytes} [Mon Jul 18 02:18:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:18:34.322135 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1a40da-eed9-4709-a30b-b3361372df2e None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:18:34.361136 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:18:34.420165 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1a40da-eed9-4709-a30b-b3361372df2e None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:18:34.453619 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae29fc6-eb4f-44cf-ada5-bbb391834449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:18:34.454317 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae29fc6-eb4f-44cf-ada5-bbb391834449 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:18:34.454605 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae29fc6-eb4f-44cf-ada5-bbb391834449 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:18:34.454884 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae29fc6-eb4f-44cf-ada5-bbb391834449 None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:18:34.455641 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae29fc6-eb4f-44cf-ada5-bbb391834449 None admin] RBAC: Adding query filter params (name=service) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:18:34.455935 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae29fc6-eb4f-44cf-ada5-bbb391834449 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:18:34.457311 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae29fc6-eb4f-44cf-ada5-bbb391834449 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:18:34.465068 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 599/1209] 10.222.0.66 () {56 vars in 1103 bytes} [Mon Jul 18 02:18:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:18:34.471146 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac1057a-779a-4a84-a8d1-ffe61fcf050d None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:18:34.520973 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:18:34.572607 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac1057a-779a-4a84-a8d1-ffe61fcf050d None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:18:34.607863 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f37ce5-d0a5-43f2-a58f-a19a690d3711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:18:34.608665 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f37ce5-d0a5-43f2-a58f-a19a690d3711 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:18:34.608947 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f37ce5-d0a5-43f2-a58f-a19a690d3711 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:18:34.609222 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f37ce5-d0a5-43f2-a58f-a19a690d3711 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:18:34.620668 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f37ce5-d0a5-43f2-a58f-a19a690d3711 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:18:34.622116 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f37ce5-d0a5-43f2-a58f-a19a690d3711 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:18:34.639794 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-e4f37ce5-d0a5-43f2-a58f-a19a690d3711 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:18:34.639794 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:18:34.639794 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:18:34.639794 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:18:34.639794 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:18:34.639794 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:18:34.639794 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:18:34.639794 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:18:34.639794 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:18:34.639794 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:18:34.639794 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:18:34.639794 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:18:34.639794 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 02:18:34.639794 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 02:18:34.639794 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 02:18:34.640873 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 02:18:34.640873 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:18:34.640873 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:18:34.640873 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:18:34.640873 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:18:34.640873 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:18:34.640873 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:18:34.640873 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:18:34.640873 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:18:34.640873 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:18:34.640873 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:18:34.640873 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:18:34.640873 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:18:34.640873 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:18:34.640873 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:18:34.640873 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:18:34.640873 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:18:34.641836 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 02:18:34.641836 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 02:18:34.641836 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:18:34.641836 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:18:34.641836 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 611/1210] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 18 02:18:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 02:18:34.646819 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae29fc6-eb4f-44cf-ada5-bbb391834449 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:18:34.745449 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:18:34.748324 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae29fc6-eb4f-44cf-ada5-bbb391834449 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:18:34.783812 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652df09b-9500-4abe-8020-745f1d527372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:18:34.784472 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652df09b-9500-4abe-8020-745f1d527372 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:18:34.784800 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652df09b-9500-4abe-8020-745f1d527372 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:18:34.785078 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652df09b-9500-4abe-8020-745f1d527372 None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:18:34.786056 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652df09b-9500-4abe-8020-745f1d527372 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:18:34.786399 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652df09b-9500-4abe-8020-745f1d527372 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:18:34.787779 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652df09b-9500-4abe-8020-745f1d527372 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:18:34.800515 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 600/1211] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 18 02:18:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:18:34.807096 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f37ce5-d0a5-43f2-a58f-a19a690d3711 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:18:34.906376 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f37ce5-d0a5-43f2-a58f-a19a690d3711 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:18:34.907883 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:18:34.939866 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a674d7f-6d9c-4d5c-bdf0-50e224794d40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:18:34.940655 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a674d7f-6d9c-4d5c-bdf0-50e224794d40 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:18:34.940939 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a674d7f-6d9c-4d5c-bdf0-50e224794d40 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:18:34.941212 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a674d7f-6d9c-4d5c-bdf0-50e224794d40 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:18:34.949362 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a674d7f-6d9c-4d5c-bdf0-50e224794d40 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:18:34.950853 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a674d7f-6d9c-4d5c-bdf0-50e224794d40 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:18:34.964984 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-0a674d7f-6d9c-4d5c-bdf0-50e224794d40 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 18 02:18:34.964984 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:18:34.964984 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:18:34.964984 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:18:34.964984 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:18:34.964984 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:18:34.964984 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:18:34.964984 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:18:34.964984 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:18:34.964984 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:18:34.964984 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:18:34.964984 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:18:34.964984 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 02:18:34.964984 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 02:18:34.964984 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 02:18:34.964984 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 02:18:34.969495 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:18:34.969495 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:18:34.969495 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 02:18:34.969495 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 02:18:34.969495 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 02:18:34.969495 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 02:18:34.969495 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:18:34.969495 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:18:34.969495 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:18:34.969495 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:18:34.969495 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:18:34.969495 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:18:34.969495 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:18:34.969495 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:18:34.969495 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:18:34.969495 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:18:34.969495 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:18:34.969495 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:18:34.970734 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:18:34.970734 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:18:34.970734 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 02:18:34.970734 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 02:18:34.970734 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 02:18:34.970734 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 02:18:34.970734 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 02:18:34.970734 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 02:18:34.970734 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 18 02:18:34.970734 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:18:34.970734 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 612/1212] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 18 02:18:34 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:18:34.972149 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652df09b-9500-4abe-8020-745f1d527372 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:18:35.074788 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652df09b-9500-4abe-8020-745f1d527372 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:18:35.109001 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8e184b-909f-4ba8-93db-27d8dd8c2e47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:18:35.109673 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8e184b-909f-4ba8-93db-27d8dd8c2e47 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:18:35.109958 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8e184b-909f-4ba8-93db-27d8dd8c2e47 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:18:35.110267 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8e184b-909f-4ba8-93db-27d8dd8c2e47 None admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:18:35.111093 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8e184b-909f-4ba8-93db-27d8dd8c2e47 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:18:35.111403 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8e184b-909f-4ba8-93db-27d8dd8c2e47 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:18:35.112825 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8e184b-909f-4ba8-93db-27d8dd8c2e47 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:18:35.129952 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 601/1213] 10.222.0.66 () {56 vars in 1143 bytes} [Mon Jul 18 02:18:34 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:18:35.135817 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a674d7f-6d9c-4d5c-bdf0-50e224794d40 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:18:35.235921 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a674d7f-6d9c-4d5c-bdf0-50e224794d40 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:18:35.268155 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2368fc-29b6-4dee-9dec-4e8607652ae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:18:35.269363 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2368fc-29b6-4dee-9dec-4e8607652ae5 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:18:35.269363 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2368fc-29b6-4dee-9dec-4e8607652ae5 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:18:35.269363 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2368fc-29b6-4dee-9dec-4e8607652ae5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:18:35.299490 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:18:35.336769 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2368fc-29b6-4dee-9dec-4e8607652ae5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:18:35.338281 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2368fc-29b6-4dee-9dec-4e8607652ae5 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:18:35.435884 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 613/1214] 10.222.0.66 () {58 vars in 1282 bytes} [Mon Jul 18 02:18:35 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 02:18:35.441944 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8e184b-909f-4ba8-93db-27d8dd8c2e47 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:18:35.462200 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:18:35.541193 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8e184b-909f-4ba8-93db-27d8dd8c2e47 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:18:35.574306 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8b6600-a295-45ee-bdae-30f203fa88ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:18:35.575100 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8b6600-a295-45ee-bdae-30f203fa88ca None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:18:35.575384 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8b6600-a295-45ee-bdae-30f203fa88ca None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:18:35.575660 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8b6600-a295-45ee-bdae-30f203fa88ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:18:35.586607 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8b6600-a295-45ee-bdae-30f203fa88ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:18:35.588064 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8b6600-a295-45ee-bdae-30f203fa88ca None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:18:35.605492 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-6c8b6600-a295-45ee-bdae-30f203fa88ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:18:35.605492 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:18:35.605492 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:18:35.605492 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:18:35.605492 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:18:35.605492 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:18:35.605492 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:18:35.605492 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:18:35.605492 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:18:35.605492 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:18:35.605492 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:18:35.605492 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:18:35.605492 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 02:18:35.605492 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 02:18:35.605492 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 02:18:35.606631 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 02:18:35.606631 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:18:35.606631 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:18:35.606631 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:18:35.606631 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:18:35.606631 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:18:35.606631 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:18:35.606631 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:18:35.606631 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:18:35.606631 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:18:35.606631 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:18:35.606631 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:18:35.606631 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:18:35.606631 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:18:35.606631 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:18:35.606631 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:18:35.606631 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:18:35.607645 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 02:18:35.607645 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 02:18:35.607645 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:18:35.607645 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:18:35.607645 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 602/1215] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 18 02:18:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:18:35.612570 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2368fc-29b6-4dee-9dec-4e8607652ae5 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:18:35.674221 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:18:35.712264 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2368fc-29b6-4dee-9dec-4e8607652ae5 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:18:35.746657 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dda975-e762-46b1-ac55-e48649c4a630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:18:35.747302 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dda975-e762-46b1-ac55-e48649c4a630 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:18:35.747587 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dda975-e762-46b1-ac55-e48649c4a630 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:18:35.747869 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dda975-e762-46b1-ac55-e48649c4a630 None admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:18:35.748582 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dda975-e762-46b1-ac55-e48649c4a630 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:18:35.748888 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dda975-e762-46b1-ac55-e48649c4a630 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:18:35.750281 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dda975-e762-46b1-ac55-e48649c4a630 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:18:35.762078 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 614/1216] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 18 02:18:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:18:35.767941 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8b6600-a295-45ee-bdae-30f203fa88ca None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:18:35.837640 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:18:35.867756 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8b6600-a295-45ee-bdae-30f203fa88ca None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:18:35.901732 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4666e5-4370-4ed4-960e-77f3a2826e7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:18:35.902569 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4666e5-4370-4ed4-960e-77f3a2826e7e None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:18:35.902859 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4666e5-4370-4ed4-960e-77f3a2826e7e None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:18:35.903140 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4666e5-4370-4ed4-960e-77f3a2826e7e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:18:35.909801 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4666e5-4370-4ed4-960e-77f3a2826e7e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:18:35.911856 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4666e5-4370-4ed4-960e-77f3a2826e7e None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:18:35.927844 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-5a4666e5-4370-4ed4-960e-77f3a2826e7e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 02:18:35.927844 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:18:35.927844 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:18:35.927844 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:18:35.927844 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:18:35.927844 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:18:35.927844 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:18:35.927844 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:18:35.927844 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:18:35.927844 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:18:35.927844 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:18:35.927844 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:18:35.927844 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 02:18:35.927844 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 02:18:35.927844 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 02:18:35.928936 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 02:18:35.928936 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:18:35.928936 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:18:35.928936 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:18:35.928936 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:18:35.928936 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:18:35.928936 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:18:35.928936 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:18:35.928936 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:18:35.928936 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:18:35.928936 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:18:35.928936 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:18:35.928936 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:18:35.928936 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:18:35.928936 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:18:35.928936 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:18:35.928936 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:18:35.934983 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 02:18:35.934983 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 02:18:35.934983 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 02:18:35.934983 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 02:18:35.934983 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 02:18:35.934983 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 02:18:35.934983 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 02:18:35.934983 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:18:35.934983 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 603/1217] 10.222.0.66 () {56 vars in 1111 bytes} [Mon Jul 18 02:18:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:18:35.935621 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dda975-e762-46b1-ac55-e48649c4a630 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:18:36.033761 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dda975-e762-46b1-ac55-e48649c4a630 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:18:36.056322 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:18:36.067380 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffee975-de6e-49d9-8cfa-03040b7a6317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:18:36.070729 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffee975-de6e-49d9-8cfa-03040b7a6317 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:18:36.070729 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffee975-de6e-49d9-8cfa-03040b7a6317 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:18:36.070729 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffee975-de6e-49d9-8cfa-03040b7a6317 None admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:18:36.070729 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffee975-de6e-49d9-8cfa-03040b7a6317 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:18:36.070729 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffee975-de6e-49d9-8cfa-03040b7a6317 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:18:36.071852 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffee975-de6e-49d9-8cfa-03040b7a6317 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:18:36.083539 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 615/1218] 10.222.0.66 () {56 vars in 1148 bytes} [Mon Jul 18 02:18:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:18:36.089674 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4666e5-4370-4ed4-960e-77f3a2826e7e None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:18:36.187965 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4666e5-4370-4ed4-960e-77f3a2826e7e None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:18:36.216300 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:18:36.221910 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb6e542-4ce1-40b6-baf9-58709eca01fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:18:36.222583 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb6e542-4ce1-40b6-baf9-58709eca01fe None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:18:36.222871 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb6e542-4ce1-40b6-baf9-58709eca01fe None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:18:36.223151 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb6e542-4ce1-40b6-baf9-58709eca01fe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:18:36.223970 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb6e542-4ce1-40b6-baf9-58709eca01fe None admin] RBAC: Adding query filter params (role.id=eea94daf3312421cb12aa8abfa73160e, scope.project.id=76f7c0ae36ae49b09dcae4c3e9a875f4, user.id=88936b31709b429f8b6777c673ed7b8a) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:18:36.224284 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb6e542-4ce1-40b6-baf9-58709eca01fe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:18:36.225718 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb6e542-4ce1-40b6-baf9-58709eca01fe None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:18:36.231586 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 604/1219] 10.222.0.66 () {56 vars in 1374 bytes} [Mon Jul 18 02:18:36 2022] GET /identity/v3/role_assignments?user.id=88936b31709b429f8b6777c673ed7b8a&scope.project.id=76f7c0ae36ae49b09dcae4c3e9a875f4&role.id=eea94daf3312421cb12aa8abfa73160e => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:18:38.204743 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a43172-4c65-4fa6-a473-b498b1c8d06e None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:18:38.205349 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a43172-4c65-4fa6-a473-b498b1c8d06e None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:18:38.205349 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a43172-4c65-4fa6-a473-b498b1c8d06e None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:18:38.206792 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 616/1220] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:18:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:18:38.212767 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf6f7cb-952e-436b-8d87-fb033ef966cc None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:18:38.213223 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf6f7cb-952e-436b-8d87-fb033ef966cc None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:18:38.213320 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf6f7cb-952e-436b-8d87-fb033ef966cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:18:38.230634 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-baf6f7cb-952e-436b-8d87-fb033ef966cc None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:18:38.231841 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-baf6f7cb-952e-436b-8d87-fb033ef966cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:18:38.300638 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 605/1221] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:18:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 02:18:38.322913 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a43172-4c65-4fa6-a473-b498b1c8d06e None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:18:38.424132 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a43172-4c65-4fa6-a473-b498b1c8d06e None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:18:38.458778 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69caf5b-9e62-4c12-b4a1-45220eb33a42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:18:38.459495 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69caf5b-9e62-4c12-b4a1-45220eb33a42 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:18:38.459776 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69caf5b-9e62-4c12-b4a1-45220eb33a42 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:18:38.460047 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69caf5b-9e62-4c12-b4a1-45220eb33a42 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:18:38.470510 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69caf5b-9e62-4c12-b4a1-45220eb33a42 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 02:18:38.478173 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-d69caf5b-9e62-4c12-b4a1-45220eb33a42 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 02:18:38.478173 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:18:38.478173 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:18:38.478173 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:18:38.478173 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:18:38.478173 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:18:38.478173 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:18:38.478173 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:18:38.478173 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:18:38.478173 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:18:38.478173 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:18:38.478173 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:18:38.478173 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 02:18:38.478173 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 02:18:38.478173 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 02:18:38.478173 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 02:18:38.482707 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 02:18:38.482707 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 02:18:38.482707 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 02:18:38.482707 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 02:18:38.482707 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:18:38.482707 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:18:38.482707 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:18:38.482707 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:18:38.482707 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:18:38.482707 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:18:38.482707 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:18:38.482707 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:18:38.482707 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:18:38.482707 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:18:38.482707 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:18:38.482707 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:18:38.483749 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:18:38.483749 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:18:38.483749 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:18:38.483749 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:18:38.483749 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 02:18:38.483749 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 02:18:38.483749 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 02:18:38.483749 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 02:18:38.483749 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 02:18:38.483749 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 02:18:38.483749 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 02:18:38.483749 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:18:38.483749 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 617/1222] 10.222.0.66 () {56 vars in 1102 bytes} [Mon Jul 18 02:18:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:18:38.485839 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf6f7cb-952e-436b-8d87-fb033ef966cc None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:18:38.585981 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf6f7cb-952e-436b-8d87-fb033ef966cc None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:18:38.613073 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:18:38.625515 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6a0e08-22cd-468a-96b7-418ace39dabb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:18:38.627189 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6a0e08-22cd-468a-96b7-418ace39dabb None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:18:38.628477 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6a0e08-22cd-468a-96b7-418ace39dabb None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:18:38.632974 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6a0e08-22cd-468a-96b7-418ace39dabb None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:18:38.632974 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6a0e08-22cd-468a-96b7-418ace39dabb None admin] RBAC: Adding query filter params (name=service) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:18:38.632974 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6a0e08-22cd-468a-96b7-418ace39dabb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:18:38.632974 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6a0e08-22cd-468a-96b7-418ace39dabb None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:18:38.640271 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 606/1223] 10.222.0.66 () {56 vars in 1103 bytes} [Mon Jul 18 02:18:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:18:38.646108 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69caf5b-9e62-4c12-b4a1-45220eb33a42 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:18:38.693378 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:18:38.753140 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69caf5b-9e62-4c12-b4a1-45220eb33a42 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:18:38.795496 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e464aa-60e7-42ad-9775-73d3353f1c97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:18:38.796291 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e464aa-60e7-42ad-9775-73d3353f1c97 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:18:38.796583 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e464aa-60e7-42ad-9775-73d3353f1c97 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:18:38.796857 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e464aa-60e7-42ad-9775-73d3353f1c97 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:18:38.808534 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e464aa-60e7-42ad-9775-73d3353f1c97 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:18:38.810020 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e464aa-60e7-42ad-9775-73d3353f1c97 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:18:38.829010 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-06e464aa-60e7-42ad-9775-73d3353f1c97 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:18:38.829010 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:18:38.829010 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:18:38.829010 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:18:38.829010 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:18:38.829010 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:18:38.829010 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:18:38.829010 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:18:38.829010 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:18:38.829010 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:18:38.829010 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:18:38.829010 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:18:38.829010 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 02:18:38.829010 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 02:18:38.829010 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 02:18:38.830652 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 02:18:38.830652 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:18:38.830652 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:18:38.830652 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:18:38.830652 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:18:38.830652 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:18:38.830652 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:18:38.830652 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:18:38.830652 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:18:38.830652 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:18:38.830652 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:18:38.830652 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:18:38.830652 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:18:38.830652 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:18:38.830652 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:18:38.830652 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:18:38.830652 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:18:38.831784 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 02:18:38.831784 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 02:18:38.831784 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:18:38.831784 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:18:38.831784 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 618/1224] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 18 02:18:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:18:38.836999 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6a0e08-22cd-468a-96b7-418ace39dabb None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:18:38.959915 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6a0e08-22cd-468a-96b7-418ace39dabb None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:18:39.000210 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032ee27c-da02-4c72-8e73-b5cf6417ba35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:18:39.000875 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032ee27c-da02-4c72-8e73-b5cf6417ba35 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:18:39.001160 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032ee27c-da02-4c72-8e73-b5cf6417ba35 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:18:39.001437 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032ee27c-da02-4c72-8e73-b5cf6417ba35 None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:18:39.002183 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032ee27c-da02-4c72-8e73-b5cf6417ba35 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:18:39.002496 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032ee27c-da02-4c72-8e73-b5cf6417ba35 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:18:39.005627 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032ee27c-da02-4c72-8e73-b5cf6417ba35 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:18:39.016384 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 607/1225] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 18 02:18:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:18:39.022498 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e464aa-60e7-42ad-9775-73d3353f1c97 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:18:39.124344 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:18:39.134345 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e464aa-60e7-42ad-9775-73d3353f1c97 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:18:39.171829 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b1e2ea-69af-4383-aa6a-445912602a4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:18:39.172642 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b1e2ea-69af-4383-aa6a-445912602a4e None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:18:39.172932 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b1e2ea-69af-4383-aa6a-445912602a4e None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:18:39.173203 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b1e2ea-69af-4383-aa6a-445912602a4e None admin] PATH_INFO: `/v3/users/placement` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:18:39.182201 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b1e2ea-69af-4383-aa6a-445912602a4e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:18:39.183661 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b1e2ea-69af-4383-aa6a-445912602a4e None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:18:39.201766 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-98b1e2ea-69af-4383-aa6a-445912602a4e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 18 02:18:39.201766 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:18:39.201766 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:18:39.201766 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:18:39.201766 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:18:39.201766 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:18:39.201766 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:18:39.201766 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:18:39.201766 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:18:39.201766 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:18:39.201766 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:18:39.201766 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:18:39.201766 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 02:18:39.201766 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 02:18:39.201766 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 02:18:39.201766 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 02:18:39.203018 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:18:39.203018 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:18:39.203018 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 02:18:39.203018 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 02:18:39.203018 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 02:18:39.203018 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 02:18:39.203018 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:18:39.203018 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:18:39.203018 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:18:39.203018 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:18:39.203018 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:18:39.203018 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:18:39.203018 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:18:39.203018 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:18:39.203018 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:18:39.203018 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:18:39.203018 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:18:39.203018 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:18:39.204458 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:18:39.204458 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:18:39.204458 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 02:18:39.204458 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 02:18:39.204458 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 02:18:39.204458 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 02:18:39.204458 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 02:18:39.204458 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 02:18:39.204458 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 18 02:18:39.204458 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:18:39.204458 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 619/1226] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 18 02:18:39 2022] GET /identity/v3/users/placement => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:18:39.209328 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032ee27c-da02-4c72-8e73-b5cf6417ba35 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:18:39.261722 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:18:39.318352 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032ee27c-da02-4c72-8e73-b5cf6417ba35 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:18:39.355734 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8321bd-4230-4ae0-9ff4-a71a11783276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:18:39.356618 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8321bd-4230-4ae0-9ff4-a71a11783276 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:18:39.356958 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8321bd-4230-4ae0-9ff4-a71a11783276 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:18:39.357756 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8321bd-4230-4ae0-9ff4-a71a11783276 None admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:18:39.358629 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8321bd-4230-4ae0-9ff4-a71a11783276 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:18:39.362049 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8321bd-4230-4ae0-9ff4-a71a11783276 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:18:39.362049 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8321bd-4230-4ae0-9ff4-a71a11783276 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:18:39.382372 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 608/1227] 10.222.0.66 () {56 vars in 1143 bytes} [Mon Jul 18 02:18:39 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:18:39.390497 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b1e2ea-69af-4383-aa6a-445912602a4e None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:18:39.492325 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b1e2ea-69af-4383-aa6a-445912602a4e None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:18:39.526769 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a65b04-f5ef-415a-970d-a5ab7a8c3d02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:18:39.527484 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a65b04-f5ef-415a-970d-a5ab7a8c3d02 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:18:39.527765 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a65b04-f5ef-415a-970d-a5ab7a8c3d02 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:18:39.528036 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a65b04-f5ef-415a-970d-a5ab7a8c3d02 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:18:39.540613 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a65b04-f5ef-415a-970d-a5ab7a8c3d02 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:18:39.542050 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a65b04-f5ef-415a-970d-a5ab7a8c3d02 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:18:39.559776 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-b4a65b04-f5ef-415a-970d-a5ab7a8c3d02 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:18:39.559776 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:18:39.559776 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:18:39.559776 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:18:39.559776 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:18:39.559776 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:18:39.559776 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:18:39.559776 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:18:39.559776 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:18:39.559776 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:18:39.559776 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:18:39.559776 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:18:39.559776 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 02:18:39.559776 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 02:18:39.559776 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 02:18:39.560962 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 02:18:39.560962 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:18:39.560962 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:18:39.560962 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:18:39.560962 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:18:39.560962 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:18:39.560962 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:18:39.560962 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:18:39.560962 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:18:39.560962 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:18:39.560962 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:18:39.560962 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:18:39.560962 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:18:39.560962 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:18:39.560962 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:18:39.560962 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:18:39.560962 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:18:39.562157 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 02:18:39.562157 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 02:18:39.562157 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:18:39.562157 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:18:39.562157 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 620/1228] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 18 02:18:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 02:18:39.566993 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8321bd-4230-4ae0-9ff4-a71a11783276 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:18:39.673336 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8321bd-4230-4ae0-9ff4-a71a11783276 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:18:39.675659 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:18:39.705789 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db52392-3da8-453a-92d9-9cf5d3098c6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:18:39.706471 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db52392-3da8-453a-92d9-9cf5d3098c6e None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:18:39.706755 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db52392-3da8-453a-92d9-9cf5d3098c6e None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:18:39.707026 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db52392-3da8-453a-92d9-9cf5d3098c6e None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:18:39.707731 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db52392-3da8-453a-92d9-9cf5d3098c6e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:18:39.708032 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db52392-3da8-453a-92d9-9cf5d3098c6e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:18:39.709430 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db52392-3da8-453a-92d9-9cf5d3098c6e None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:18:39.721456 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 609/1229] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 18 02:18:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:18:39.728059 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a65b04-f5ef-415a-970d-a5ab7a8c3d02 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:18:39.737294 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:18:39.826100 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a65b04-f5ef-415a-970d-a5ab7a8c3d02 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:18:39.859946 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4724e0-b421-4b58-8b88-6c3253cc5864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:18:39.860722 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4724e0-b421-4b58-8b88-6c3253cc5864 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:18:39.861000 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4724e0-b421-4b58-8b88-6c3253cc5864 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:18:39.861271 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4724e0-b421-4b58-8b88-6c3253cc5864 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:18:39.867856 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4724e0-b421-4b58-8b88-6c3253cc5864 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:18:39.869351 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4724e0-b421-4b58-8b88-6c3253cc5864 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:18:39.883940 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-fc4724e0-b421-4b58-8b88-6c3253cc5864 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 02:18:39.883940 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:18:39.883940 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:18:39.883940 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:18:39.883940 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:18:39.883940 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:18:39.883940 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:18:39.883940 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:18:39.883940 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:18:39.883940 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:18:39.883940 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:18:39.883940 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:18:39.883940 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 02:18:39.883940 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 02:18:39.883940 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 02:18:39.885025 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 02:18:39.885025 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:18:39.885025 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:18:39.885025 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:18:39.885025 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:18:39.885025 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:18:39.885025 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:18:39.885025 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:18:39.885025 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:18:39.885025 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:18:39.885025 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:18:39.885025 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:18:39.885025 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:18:39.885025 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:18:39.885025 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:18:39.885025 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:18:39.885025 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:18:39.886171 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 02:18:39.886171 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 02:18:39.886171 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 02:18:39.886171 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 02:18:39.886171 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 02:18:39.886171 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 02:18:39.886171 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 02:18:39.886171 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:18:39.886171 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 621/1230] 10.222.0.66 () {56 vars in 1111 bytes} [Mon Jul 18 02:18:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:18:39.891138 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db52392-3da8-453a-92d9-9cf5d3098c6e None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:18:39.991504 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db52392-3da8-453a-92d9-9cf5d3098c6e None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:18:40.024883 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19db2b77-5c85-4655-b9d8-1565ab24be62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:18:40.025544 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19db2b77-5c85-4655-b9d8-1565ab24be62 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:18:40.025832 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19db2b77-5c85-4655-b9d8-1565ab24be62 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:18:40.026142 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19db2b77-5c85-4655-b9d8-1565ab24be62 None admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:18:40.026954 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19db2b77-5c85-4655-b9d8-1565ab24be62 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:18:40.027267 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19db2b77-5c85-4655-b9d8-1565ab24be62 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:18:40.028732 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19db2b77-5c85-4655-b9d8-1565ab24be62 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:18:40.041129 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 610/1231] 10.222.0.66 () {56 vars in 1148 bytes} [Mon Jul 18 02:18:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:18:40.047332 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4724e0-b421-4b58-8b88-6c3253cc5864 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:18:40.111464 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:18:40.146314 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4724e0-b421-4b58-8b88-6c3253cc5864 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:18:40.180906 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4557038-5ee3-4399-b17d-257172c11886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:18:40.181776 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4557038-5ee3-4399-b17d-257172c11886 None admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:18:40.182059 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4557038-5ee3-4399-b17d-257172c11886 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:18:40.182405 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4557038-5ee3-4399-b17d-257172c11886 None admin] PATH_INFO: `/v3/projects/76f7c0ae36ae49b09dcae4c3e9a875f4/users/88936b31709b429f8b6777c673ed7b8a/roles/eea94daf3312421cb12aa8abfa73160e` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:18:40.197116 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4557038-5ee3-4399-b17d-257172c11886 None admin] RBAC: Authorizing `identity:create_grant(project_id=76f7c0ae36ae49b09dcae4c3e9a875f4, user_id=88936b31709b429f8b6777c673ed7b8a, role_id=eea94daf3312421cb12aa8abfa73160e)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:18:40.198614 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4557038-5ee3-4399-b17d-257172c11886 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:18:40.286513 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 622/1232] 10.222.0.66 () {58 vars in 1439 bytes} [Mon Jul 18 02:18:40 2022] PUT /identity/v3/projects/76f7c0ae36ae49b09dcae4c3e9a875f4/users/88936b31709b429f8b6777c673ed7b8a/roles/eea94daf3312421cb12aa8abfa73160e => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:18:42.295280 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa2de87-44b6-4dbc-b741-a14f7681a951 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:18:42.295824 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa2de87-44b6-4dbc-b741-a14f7681a951 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:18:42.295824 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa2de87-44b6-4dbc-b741-a14f7681a951 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:18:42.297217 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 611/1233] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:18:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:18:42.302798 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2affdf-8fe6-4111-b23b-0db5510e0277 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:18:42.303231 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2affdf-8fe6-4111-b23b-0db5510e0277 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:18:42.303345 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2affdf-8fe6-4111-b23b-0db5510e0277 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:18:42.319954 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-5b2affdf-8fe6-4111-b23b-0db5510e0277 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:18:42.321119 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-5b2affdf-8fe6-4111-b23b-0db5510e0277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:18:42.386156 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 623/1234] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:18:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 02:18:42.409106 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa2de87-44b6-4dbc-b741-a14f7681a951 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:18:42.434570 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:18:42.510291 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa2de87-44b6-4dbc-b741-a14f7681a951 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:18:42.544186 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b7f9cd-0a55-4e1f-adb8-9fa7acc10666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:18:42.544976 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b7f9cd-0a55-4e1f-adb8-9fa7acc10666 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:18:42.545259 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b7f9cd-0a55-4e1f-adb8-9fa7acc10666 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:18:42.545584 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b7f9cd-0a55-4e1f-adb8-9fa7acc10666 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:18:42.556354 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b7f9cd-0a55-4e1f-adb8-9fa7acc10666 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 02:18:42.564082 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-e1b7f9cd-0a55-4e1f-adb8-9fa7acc10666 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 02:18:42.564082 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:18:42.564082 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:18:42.564082 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:18:42.564082 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:18:42.564082 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:18:42.564082 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:18:42.564082 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:18:42.564082 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:18:42.564082 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:18:42.564082 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:18:42.564082 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:18:42.564082 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 02:18:42.564082 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 02:18:42.564082 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 02:18:42.564082 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 02:18:42.565180 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 02:18:42.565180 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 02:18:42.565180 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 02:18:42.565180 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 02:18:42.565180 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:18:42.565180 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:18:42.565180 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:18:42.565180 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:18:42.565180 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:18:42.565180 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:18:42.565180 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:18:42.565180 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:18:42.565180 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:18:42.565180 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:18:42.565180 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:18:42.565180 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:18:42.566284 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:18:42.566284 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:18:42.566284 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:18:42.566284 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:18:42.566284 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 02:18:42.566284 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 02:18:42.566284 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 02:18:42.566284 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 02:18:42.566284 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 02:18:42.566284 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 02:18:42.566284 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 02:18:42.566284 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:18:42.566284 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 612/1235] 10.222.0.66 () {56 vars in 1102 bytes} [Mon Jul 18 02:18:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:18:42.571474 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2affdf-8fe6-4111-b23b-0db5510e0277 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:18:42.618504 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:18:42.671716 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2affdf-8fe6-4111-b23b-0db5510e0277 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:18:42.706104 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d307ae-7f0e-45dc-b3ea-d227cda6ee08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:18:42.706768 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d307ae-7f0e-45dc-b3ea-d227cda6ee08 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:18:42.707051 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d307ae-7f0e-45dc-b3ea-d227cda6ee08 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:18:42.707325 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d307ae-7f0e-45dc-b3ea-d227cda6ee08 None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:18:42.708074 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d307ae-7f0e-45dc-b3ea-d227cda6ee08 None admin] RBAC: Adding query filter params (name=service) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:18:42.708368 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d307ae-7f0e-45dc-b3ea-d227cda6ee08 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:18:42.709724 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d307ae-7f0e-45dc-b3ea-d227cda6ee08 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:18:42.718543 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 624/1236] 10.222.0.66 () {56 vars in 1103 bytes} [Mon Jul 18 02:18:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:18:42.724600 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b7f9cd-0a55-4e1f-adb8-9fa7acc10666 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:18:42.810070 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:18:42.825505 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b7f9cd-0a55-4e1f-adb8-9fa7acc10666 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:18:42.858981 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4637ba2-e28b-4968-8a8e-f87ccaff2c18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:18:42.862574 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4637ba2-e28b-4968-8a8e-f87ccaff2c18 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:18:42.862574 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4637ba2-e28b-4968-8a8e-f87ccaff2c18 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:18:42.862574 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4637ba2-e28b-4968-8a8e-f87ccaff2c18 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:18:42.870884 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4637ba2-e28b-4968-8a8e-f87ccaff2c18 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:18:42.872327 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4637ba2-e28b-4968-8a8e-f87ccaff2c18 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:18:42.890212 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-e4637ba2-e28b-4968-8a8e-f87ccaff2c18 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:18:42.890212 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:18:42.890212 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:18:42.890212 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:18:42.890212 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:18:42.890212 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:18:42.890212 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:18:42.890212 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:18:42.890212 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:18:42.890212 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:18:42.890212 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:18:42.890212 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:18:42.890212 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 02:18:42.890212 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 02:18:42.890212 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 02:18:42.891408 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 02:18:42.891408 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:18:42.891408 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:18:42.891408 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:18:42.891408 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:18:42.891408 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:18:42.891408 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:18:42.891408 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:18:42.891408 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:18:42.891408 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:18:42.891408 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:18:42.891408 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:18:42.891408 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:18:42.891408 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:18:42.891408 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:18:42.891408 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:18:42.891408 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:18:42.892794 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 02:18:42.892794 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 02:18:42.892794 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:18:42.892794 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:18:42.892794 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 613/1237] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 18 02:18:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:18:42.897334 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d307ae-7f0e-45dc-b3ea-d227cda6ee08 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:18:42.994948 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d307ae-7f0e-45dc-b3ea-d227cda6ee08 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:18:43.003837 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:18:43.028188 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef08246-bba5-4fd5-a0dc-fd1d813d3fa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:18:43.028868 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef08246-bba5-4fd5-a0dc-fd1d813d3fa0 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:18:43.029153 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef08246-bba5-4fd5-a0dc-fd1d813d3fa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:18:43.029421 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef08246-bba5-4fd5-a0dc-fd1d813d3fa0 None admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:18:43.030168 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef08246-bba5-4fd5-a0dc-fd1d813d3fa0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:18:43.030478 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef08246-bba5-4fd5-a0dc-fd1d813d3fa0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:18:43.031861 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef08246-bba5-4fd5-a0dc-fd1d813d3fa0 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:18:43.044577 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 625/1238] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 18 02:18:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:18:43.050352 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4637ba2-e28b-4968-8a8e-f87ccaff2c18 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:18:43.148600 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4637ba2-e28b-4968-8a8e-f87ccaff2c18 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:18:43.182591 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df2ce04-9f5e-4535-a55b-55230b5dbc39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:18:43.183381 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df2ce04-9f5e-4535-a55b-55230b5dbc39 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:18:43.183676 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df2ce04-9f5e-4535-a55b-55230b5dbc39 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:18:43.183935 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df2ce04-9f5e-4535-a55b-55230b5dbc39 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:18:43.186693 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:18:43.191196 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df2ce04-9f5e-4535-a55b-55230b5dbc39 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:18:43.195921 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df2ce04-9f5e-4535-a55b-55230b5dbc39 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:18:43.207319 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-4df2ce04-9f5e-4535-a55b-55230b5dbc39 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 18 02:18:43.207319 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:18:43.207319 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:18:43.207319 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:18:43.207319 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:18:43.207319 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:18:43.207319 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:18:43.207319 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:18:43.207319 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:18:43.207319 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:18:43.207319 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:18:43.207319 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:18:43.207319 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 02:18:43.207319 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 02:18:43.207319 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 02:18:43.207319 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 02:18:43.208482 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:18:43.208482 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:18:43.208482 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 02:18:43.208482 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 02:18:43.208482 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 02:18:43.208482 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 02:18:43.208482 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:18:43.208482 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:18:43.208482 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:18:43.208482 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:18:43.208482 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:18:43.208482 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:18:43.208482 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:18:43.208482 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:18:43.208482 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:18:43.208482 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:18:43.208482 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:18:43.208482 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:18:43.209701 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:18:43.209701 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:18:43.209701 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 02:18:43.209701 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 02:18:43.209701 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 02:18:43.209701 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 02:18:43.209701 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 02:18:43.209701 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 02:18:43.209701 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 18 02:18:43.209701 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:18:43.209701 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 614/1239] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 18 02:18:43 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:18:43.214651 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef08246-bba5-4fd5-a0dc-fd1d813d3fa0 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:18:43.315144 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef08246-bba5-4fd5-a0dc-fd1d813d3fa0 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:18:43.348372 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fdee4b-3748-4714-8fa1-5d0d059cea27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:18:43.349025 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fdee4b-3748-4714-8fa1-5d0d059cea27 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:18:43.349305 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fdee4b-3748-4714-8fa1-5d0d059cea27 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:18:43.349576 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fdee4b-3748-4714-8fa1-5d0d059cea27 None admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:18:43.351031 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fdee4b-3748-4714-8fa1-5d0d059cea27 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:18:43.351558 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fdee4b-3748-4714-8fa1-5d0d059cea27 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:18:43.353036 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fdee4b-3748-4714-8fa1-5d0d059cea27 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:18:43.370738 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 626/1240] 10.222.0.66 () {56 vars in 1143 bytes} [Mon Jul 18 02:18:43 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:18:43.376447 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df2ce04-9f5e-4535-a55b-55230b5dbc39 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:18:43.477581 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df2ce04-9f5e-4535-a55b-55230b5dbc39 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:18:43.511935 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00e93d8-0394-4f03-89d0-ad17233fef9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:18:43.512587 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00e93d8-0394-4f03-89d0-ad17233fef9e None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:18:43.512872 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00e93d8-0394-4f03-89d0-ad17233fef9e None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:18:43.513235 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00e93d8-0394-4f03-89d0-ad17233fef9e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:18:43.574189 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:18:43.580545 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00e93d8-0394-4f03-89d0-ad17233fef9e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:18:43.582001 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00e93d8-0394-4f03-89d0-ad17233fef9e None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:18:43.681210 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 615/1241] 10.222.0.66 () {58 vars in 1282 bytes} [Mon Jul 18 02:18:43 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 02:18:43.687085 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fdee4b-3748-4714-8fa1-5d0d059cea27 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:18:43.714388 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:18:43.783898 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fdee4b-3748-4714-8fa1-5d0d059cea27 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:18:43.817161 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49d6b8c-fe2a-49d0-a537-a1a0ed8c9c5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:18:43.817929 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49d6b8c-fe2a-49d0-a537-a1a0ed8c9c5c None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:18:43.818240 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49d6b8c-fe2a-49d0-a537-a1a0ed8c9c5c None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:18:43.818513 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49d6b8c-fe2a-49d0-a537-a1a0ed8c9c5c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:18:43.828384 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49d6b8c-fe2a-49d0-a537-a1a0ed8c9c5c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:18:43.829811 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49d6b8c-fe2a-49d0-a537-a1a0ed8c9c5c None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:18:43.847190 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-e49d6b8c-fe2a-49d0-a537-a1a0ed8c9c5c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:18:43.847190 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:18:43.847190 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:18:43.847190 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:18:43.847190 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:18:43.847190 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:18:43.847190 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:18:43.847190 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:18:43.847190 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:18:43.847190 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:18:43.847190 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:18:43.847190 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:18:43.847190 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 02:18:43.847190 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 02:18:43.847190 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 02:18:43.848392 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 02:18:43.848392 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:18:43.848392 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:18:43.848392 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:18:43.848392 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:18:43.848392 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:18:43.848392 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:18:43.848392 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:18:43.848392 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:18:43.848392 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:18:43.848392 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:18:43.848392 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:18:43.848392 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:18:43.848392 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:18:43.848392 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:18:43.848392 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:18:43.848392 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:18:43.849296 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 02:18:43.849296 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 02:18:43.849296 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:18:43.849296 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:18:43.849296 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 627/1242] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 18 02:18:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:18:43.854870 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00e93d8-0394-4f03-89d0-ad17233fef9e None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:18:43.949982 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:18:43.955878 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00e93d8-0394-4f03-89d0-ad17233fef9e None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:18:43.989208 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3009957c-40a9-484c-a1e9-814f56c01245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:18:43.989864 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3009957c-40a9-484c-a1e9-814f56c01245 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:18:43.990179 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3009957c-40a9-484c-a1e9-814f56c01245 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:18:43.990617 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3009957c-40a9-484c-a1e9-814f56c01245 None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:18:43.991758 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3009957c-40a9-484c-a1e9-814f56c01245 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:18:43.992064 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3009957c-40a9-484c-a1e9-814f56c01245 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:18:43.993449 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3009957c-40a9-484c-a1e9-814f56c01245 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:18:44.004649 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 616/1243] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 18 02:18:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:18:44.010461 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49d6b8c-fe2a-49d0-a537-a1a0ed8c9c5c None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:18:44.089184 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:18:44.110313 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49d6b8c-fe2a-49d0-a537-a1a0ed8c9c5c None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:18:44.142333 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8527d97-ee6b-4b77-9bf4-bdfc80a8779c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:18:44.143111 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8527d97-ee6b-4b77-9bf4-bdfc80a8779c None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:18:44.143390 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8527d97-ee6b-4b77-9bf4-bdfc80a8779c None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:18:44.143661 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8527d97-ee6b-4b77-9bf4-bdfc80a8779c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:18:44.150941 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8527d97-ee6b-4b77-9bf4-bdfc80a8779c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:18:44.152365 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8527d97-ee6b-4b77-9bf4-bdfc80a8779c None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:18:44.166717 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-e8527d97-ee6b-4b77-9bf4-bdfc80a8779c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 02:18:44.166717 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:18:44.166717 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:18:44.166717 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:18:44.166717 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:18:44.166717 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:18:44.166717 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:18:44.166717 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:18:44.166717 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:18:44.166717 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:18:44.166717 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:18:44.166717 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:18:44.166717 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 02:18:44.166717 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 02:18:44.166717 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 02:18:44.167849 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 02:18:44.167849 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:18:44.167849 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:18:44.167849 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:18:44.167849 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:18:44.167849 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:18:44.167849 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:18:44.167849 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:18:44.167849 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:18:44.167849 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:18:44.167849 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:18:44.167849 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:18:44.167849 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:18:44.167849 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:18:44.167849 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:18:44.167849 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:18:44.167849 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:18:44.169152 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 02:18:44.169152 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 02:18:44.169152 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 02:18:44.169152 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 02:18:44.169152 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 02:18:44.169152 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 02:18:44.169152 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 02:18:44.169152 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:18:44.169152 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 628/1244] 10.222.0.66 () {56 vars in 1111 bytes} [Mon Jul 18 02:18:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:18:44.173974 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3009957c-40a9-484c-a1e9-814f56c01245 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:18:44.273955 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3009957c-40a9-484c-a1e9-814f56c01245 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:18:44.307041 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8ce1ee-748f-412c-a73d-32f5a927fc92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:18:44.307699 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8ce1ee-748f-412c-a73d-32f5a927fc92 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:18:44.308003 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8ce1ee-748f-412c-a73d-32f5a927fc92 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:18:44.308280 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8ce1ee-748f-412c-a73d-32f5a927fc92 None admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:18:44.309061 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8ce1ee-748f-412c-a73d-32f5a927fc92 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:18:44.309373 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8ce1ee-748f-412c-a73d-32f5a927fc92 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:18:44.310841 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8ce1ee-748f-412c-a73d-32f5a927fc92 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:18:44.323143 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 617/1245] 10.222.0.66 () {56 vars in 1148 bytes} [Mon Jul 18 02:18:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 02:18:44.329322 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8527d97-ee6b-4b77-9bf4-bdfc80a8779c None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:18:44.428280 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8527d97-ee6b-4b77-9bf4-bdfc80a8779c None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:18:44.462509 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836735b4-d6f7-4920-9006-441c1150b05b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:18:44.463174 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836735b4-d6f7-4920-9006-441c1150b05b None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:18:44.463462 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836735b4-d6f7-4920-9006-441c1150b05b None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:18:44.463743 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836735b4-d6f7-4920-9006-441c1150b05b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:18:44.464564 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836735b4-d6f7-4920-9006-441c1150b05b None admin] RBAC: Adding query filter params (role.id=eea94daf3312421cb12aa8abfa73160e, scope.project.id=76f7c0ae36ae49b09dcae4c3e9a875f4, user.id=88936b31709b429f8b6777c673ed7b8a) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:18:44.464876 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836735b4-d6f7-4920-9006-441c1150b05b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:18:44.466330 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836735b4-d6f7-4920-9006-441c1150b05b None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:18:44.472214 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 629/1246] 10.222.0.66 () {56 vars in 1374 bytes} [Mon Jul 18 02:18:44 2022] GET /identity/v3/role_assignments?user.id=88936b31709b429f8b6777c673ed7b8a&scope.project.id=76f7c0ae36ae49b09dcae4c3e9a875f4&role.id=eea94daf3312421cb12aa8abfa73160e => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:18:46.526459 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71b5011-c791-4674-ab00-039f38ddf7d2 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:18:46.527034 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71b5011-c791-4674-ab00-039f38ddf7d2 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:18:46.527034 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71b5011-c791-4674-ab00-039f38ddf7d2 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:18:46.528408 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 618/1247] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:18:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:18:46.533578 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdcd092-d309-42bf-b808-9dec67782012 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:18:46.533985 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdcd092-d309-42bf-b808-9dec67782012 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:18:46.534178 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdcd092-d309-42bf-b808-9dec67782012 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:18:46.540193 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:18:46.551844 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-8fdcd092-d309-42bf-b808-9dec67782012 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:18:46.553023 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-8fdcd092-d309-42bf-b808-9dec67782012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:18:46.619106 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 630/1248] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:18:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 02:18:46.642597 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71b5011-c791-4674-ab00-039f38ddf7d2 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:18:46.661704 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:18:46.743451 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71b5011-c791-4674-ab00-039f38ddf7d2 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:18:46.776739 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949d7b58-26c1-4c31-89c5-6d67cdd1f997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:18:46.777464 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949d7b58-26c1-4c31-89c5-6d67cdd1f997 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:18:46.777744 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949d7b58-26c1-4c31-89c5-6d67cdd1f997 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:18:46.778021 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949d7b58-26c1-4c31-89c5-6d67cdd1f997 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:18:46.789159 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949d7b58-26c1-4c31-89c5-6d67cdd1f997 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 02:18:46.796783 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-949d7b58-26c1-4c31-89c5-6d67cdd1f997 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 02:18:46.796783 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:18:46.796783 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:18:46.796783 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:18:46.796783 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:18:46.796783 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:18:46.796783 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:18:46.796783 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:18:46.796783 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:18:46.796783 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:18:46.796783 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:18:46.796783 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:18:46.796783 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 02:18:46.796783 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 02:18:46.796783 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 02:18:46.796783 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 02:18:46.797884 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 02:18:46.797884 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 02:18:46.797884 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 02:18:46.797884 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 02:18:46.797884 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:18:46.797884 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:18:46.797884 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:18:46.797884 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:18:46.797884 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:18:46.797884 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:18:46.797884 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:18:46.797884 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:18:46.797884 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:18:46.797884 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:18:46.797884 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:18:46.797884 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:18:46.799053 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:18:46.799053 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:18:46.799053 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:18:46.799053 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:18:46.799053 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 02:18:46.799053 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 02:18:46.799053 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 02:18:46.799053 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 02:18:46.799053 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 02:18:46.799053 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 02:18:46.799053 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 02:18:46.799053 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:18:46.799053 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 619/1249] 10.222.0.66 () {56 vars in 1096 bytes} [Mon Jul 18 02:18:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:18:46.804066 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdcd092-d309-42bf-b808-9dec67782012 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:18:46.901525 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdcd092-d309-42bf-b808-9dec67782012 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:18:46.929975 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:18:46.934619 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dcbee1-3d36-4f89-adaa-d09cbee0925d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:18:46.935275 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dcbee1-3d36-4f89-adaa-d09cbee0925d None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:18:46.935560 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dcbee1-3d36-4f89-adaa-d09cbee0925d None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:18:46.935831 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dcbee1-3d36-4f89-adaa-d09cbee0925d None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:18:46.936653 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69dcbee1-3d36-4f89-adaa-d09cbee0925d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:18:46.936957 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69dcbee1-3d36-4f89-adaa-d09cbee0925d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:18:46.938334 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69dcbee1-3d36-4f89-adaa-d09cbee0925d None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:18:46.946790 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 631/1250] 10.222.0.66 () {56 vars in 1099 bytes} [Mon Jul 18 02:18:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:18:46.952807 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949d7b58-26c1-4c31-89c5-6d67cdd1f997 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:18:47.030405 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:18:47.052248 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949d7b58-26c1-4c31-89c5-6d67cdd1f997 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:18:47.086094 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140398fd-c7a9-4f78-9606-e366ed41bf3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:18:47.086893 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140398fd-c7a9-4f78-9606-e366ed41bf3e None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:18:47.087183 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140398fd-c7a9-4f78-9606-e366ed41bf3e None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:18:47.087459 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140398fd-c7a9-4f78-9606-e366ed41bf3e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:18:47.097308 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140398fd-c7a9-4f78-9606-e366ed41bf3e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:18:47.098815 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140398fd-c7a9-4f78-9606-e366ed41bf3e None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:18:47.117946 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-140398fd-c7a9-4f78-9606-e366ed41bf3e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:18:47.117946 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:18:47.117946 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:18:47.117946 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:18:47.117946 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:18:47.117946 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:18:47.117946 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:18:47.117946 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:18:47.117946 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:18:47.117946 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:18:47.117946 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:18:47.117946 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:18:47.117946 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 02:18:47.117946 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 02:18:47.117946 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 02:18:47.119076 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 02:18:47.119076 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:18:47.119076 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:18:47.119076 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:18:47.119076 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:18:47.119076 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:18:47.119076 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:18:47.119076 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:18:47.119076 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:18:47.119076 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:18:47.119076 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:18:47.119076 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:18:47.119076 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:18:47.119076 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:18:47.119076 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:18:47.119076 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:18:47.119076 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:18:47.120299 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 02:18:47.120299 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 02:18:47.120299 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:18:47.120299 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:18:47.120299 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 620/1251] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 18 02:18:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:18:47.125058 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dcbee1-3d36-4f89-adaa-d09cbee0925d None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:18:47.224220 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dcbee1-3d36-4f89-adaa-d09cbee0925d None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:18:47.259100 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e998799-9b66-446a-95e5-fe002d60a668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:18:47.259797 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e998799-9b66-446a-95e5-fe002d60a668 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:18:47.260084 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e998799-9b66-446a-95e5-fe002d60a668 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:18:47.260363 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e998799-9b66-446a-95e5-fe002d60a668 None admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:18:47.261127 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e998799-9b66-446a-95e5-fe002d60a668 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:18:47.261425 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e998799-9b66-446a-95e5-fe002d60a668 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:18:47.262882 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e998799-9b66-446a-95e5-fe002d60a668 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:18:47.274496 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 632/1252] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 18 02:18:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:18:47.280203 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140398fd-c7a9-4f78-9606-e366ed41bf3e None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:18:47.379509 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140398fd-c7a9-4f78-9606-e366ed41bf3e None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:18:47.405785 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:18:47.413813 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbf5dba-b4ca-41bd-be0f-4b1c16c431db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:18:47.419791 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbf5dba-b4ca-41bd-be0f-4b1c16c431db None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:18:47.419791 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbf5dba-b4ca-41bd-be0f-4b1c16c431db None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:18:47.419791 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbf5dba-b4ca-41bd-be0f-4b1c16c431db None admin] PATH_INFO: `/v3/users/placement` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:18:47.422252 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbf5dba-b4ca-41bd-be0f-4b1c16c431db None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:18:47.423706 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbf5dba-b4ca-41bd-be0f-4b1c16c431db None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:18:47.438929 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-9fbf5dba-b4ca-41bd-be0f-4b1c16c431db None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 18 02:18:47.438929 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:18:47.438929 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:18:47.438929 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:18:47.438929 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:18:47.438929 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:18:47.438929 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:18:47.438929 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:18:47.438929 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:18:47.438929 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:18:47.438929 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:18:47.438929 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:18:47.438929 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 02:18:47.438929 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 02:18:47.438929 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 02:18:47.438929 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 02:18:47.439957 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:18:47.439957 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:18:47.439957 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 02:18:47.439957 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 02:18:47.439957 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 02:18:47.439957 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 02:18:47.439957 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:18:47.439957 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:18:47.439957 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:18:47.439957 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:18:47.439957 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:18:47.439957 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:18:47.439957 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:18:47.439957 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:18:47.439957 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:18:47.439957 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:18:47.439957 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:18:47.439957 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:18:47.440918 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:18:47.440918 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:18:47.440918 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 02:18:47.440918 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 02:18:47.440918 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 02:18:47.440918 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 02:18:47.440918 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 02:18:47.440918 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 02:18:47.440918 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 18 02:18:47.440918 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:18:47.440918 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 621/1253] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 18 02:18:47 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:18:47.446702 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e998799-9b66-446a-95e5-fe002d60a668 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:18:47.501366 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:18:47.544848 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e998799-9b66-446a-95e5-fe002d60a668 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:18:47.578602 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc68e370-dce6-4e11-9dd6-2ed376b604a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:18:47.579260 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc68e370-dce6-4e11-9dd6-2ed376b604a7 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:18:47.579539 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc68e370-dce6-4e11-9dd6-2ed376b604a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:18:47.579810 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc68e370-dce6-4e11-9dd6-2ed376b604a7 None admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:18:47.580734 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc68e370-dce6-4e11-9dd6-2ed376b604a7 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:18:47.581044 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc68e370-dce6-4e11-9dd6-2ed376b604a7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:18:47.582493 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc68e370-dce6-4e11-9dd6-2ed376b604a7 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:18:47.599786 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 633/1254] 10.222.0.66 () {56 vars in 1143 bytes} [Mon Jul 18 02:18:47 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:18:47.606259 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbf5dba-b4ca-41bd-be0f-4b1c16c431db None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:18:47.703897 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbf5dba-b4ca-41bd-be0f-4b1c16c431db None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:18:47.737252 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03eaedfb-b015-44c5-adfa-1ccc3dc583c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:18:47.737895 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eaedfb-b015-44c5-adfa-1ccc3dc583c2 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:18:47.738209 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eaedfb-b015-44c5-adfa-1ccc3dc583c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:18:47.738487 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eaedfb-b015-44c5-adfa-1ccc3dc583c2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:18:47.792022 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:18:47.804846 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03eaedfb-b015-44c5-adfa-1ccc3dc583c2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:18:47.806322 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03eaedfb-b015-44c5-adfa-1ccc3dc583c2 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:18:47.899783 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 622/1255] 10.222.0.66 () {58 vars in 1282 bytes} [Mon Jul 18 02:18:47 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 02:18:47.905756 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc68e370-dce6-4e11-9dd6-2ed376b604a7 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:18:48.006333 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc68e370-dce6-4e11-9dd6-2ed376b604a7 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:18:48.019032 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:18:48.039806 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526cf5de-b8fa-424f-9083-a21283a855ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:18:48.040583 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526cf5de-b8fa-424f-9083-a21283a855ff None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:18:48.040865 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526cf5de-b8fa-424f-9083-a21283a855ff None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:18:48.041139 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526cf5de-b8fa-424f-9083-a21283a855ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:18:48.051369 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526cf5de-b8fa-424f-9083-a21283a855ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:18:48.052879 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526cf5de-b8fa-424f-9083-a21283a855ff None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:18:48.072120 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-526cf5de-b8fa-424f-9083-a21283a855ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:18:48.072120 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:18:48.072120 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:18:48.072120 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:18:48.072120 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:18:48.072120 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:18:48.072120 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:18:48.072120 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:18:48.072120 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:18:48.072120 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:18:48.072120 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:18:48.072120 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:18:48.072120 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 02:18:48.072120 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 02:18:48.072120 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 02:18:48.073177 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 02:18:48.073177 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:18:48.073177 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:18:48.073177 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:18:48.073177 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:18:48.073177 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:18:48.073177 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:18:48.073177 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:18:48.073177 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:18:48.073177 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:18:48.073177 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:18:48.073177 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:18:48.073177 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:18:48.073177 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:18:48.073177 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:18:48.073177 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:18:48.073177 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:18:48.074360 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 02:18:48.074360 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 02:18:48.074360 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:18:48.074360 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:18:48.074360 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 634/1256] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 18 02:18:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:18:48.079177 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03eaedfb-b015-44c5-adfa-1ccc3dc583c2 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:18:48.164287 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:18:48.175596 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03eaedfb-b015-44c5-adfa-1ccc3dc583c2 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:18:48.209823 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d04c7e-b754-4260-b778-d02c1f97601e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:18:48.210514 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d04c7e-b754-4260-b778-d02c1f97601e None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:18:48.210798 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d04c7e-b754-4260-b778-d02c1f97601e None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:18:48.211076 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d04c7e-b754-4260-b778-d02c1f97601e None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:18:48.211783 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d04c7e-b754-4260-b778-d02c1f97601e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:18:48.212091 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d04c7e-b754-4260-b778-d02c1f97601e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:18:48.213469 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d04c7e-b754-4260-b778-d02c1f97601e None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:18:48.224346 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 623/1257] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 18 02:18:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:18:48.230558 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526cf5de-b8fa-424f-9083-a21283a855ff None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:18:48.330976 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526cf5de-b8fa-424f-9083-a21283a855ff None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:18:48.364645 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5bd176-b5a9-4f9f-9f85-a42bb1a1fe95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:18:48.366044 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5bd176-b5a9-4f9f-9f85-a42bb1a1fe95 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:18:48.366576 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5bd176-b5a9-4f9f-9f85-a42bb1a1fe95 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:18:48.372695 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5bd176-b5a9-4f9f-9f85-a42bb1a1fe95 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:18:48.374698 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5bd176-b5a9-4f9f-9f85-a42bb1a1fe95 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:18:48.376148 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5bd176-b5a9-4f9f-9f85-a42bb1a1fe95 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:18:48.390110 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-0a5bd176-b5a9-4f9f-9f85-a42bb1a1fe95 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 02:18:48.390110 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:18:48.390110 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:18:48.390110 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:18:48.390110 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:18:48.390110 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:18:48.390110 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:18:48.390110 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:18:48.390110 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:18:48.390110 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:18:48.390110 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:18:48.390110 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:18:48.390110 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 02:18:48.390110 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 02:18:48.390110 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 02:18:48.391203 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 02:18:48.391203 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:18:48.391203 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:18:48.391203 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:18:48.391203 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:18:48.391203 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:18:48.391203 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:18:48.391203 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:18:48.391203 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:18:48.391203 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:18:48.391203 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:18:48.391203 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:18:48.391203 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:18:48.391203 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:18:48.391203 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:18:48.391203 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:18:48.391203 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:18:48.392294 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 02:18:48.392294 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 02:18:48.392294 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 02:18:48.392294 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 02:18:48.392294 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 02:18:48.392294 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 02:18:48.392294 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 02:18:48.392294 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:18:48.392294 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 635/1258] 10.222.0.66 () {56 vars in 1111 bytes} [Mon Jul 18 02:18:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:18:48.397207 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d04c7e-b754-4260-b778-d02c1f97601e None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:18:48.500103 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d04c7e-b754-4260-b778-d02c1f97601e None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:18:48.532643 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1d3c1d-6e16-49b8-a27e-c707aa8aa361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:18:48.533360 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1d3c1d-6e16-49b8-a27e-c707aa8aa361 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:18:48.533655 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1d3c1d-6e16-49b8-a27e-c707aa8aa361 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:18:48.533929 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1d3c1d-6e16-49b8-a27e-c707aa8aa361 None admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:18:48.534990 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1d3c1d-6e16-49b8-a27e-c707aa8aa361 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:18:48.535297 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1d3c1d-6e16-49b8-a27e-c707aa8aa361 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:18:48.536725 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1d3c1d-6e16-49b8-a27e-c707aa8aa361 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:18:48.549475 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 624/1259] 10.222.0.66 () {56 vars in 1148 bytes} [Mon Jul 18 02:18:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:18:48.557357 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5bd176-b5a9-4f9f-9f85-a42bb1a1fe95 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:18:48.583584 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:18:48.659671 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5bd176-b5a9-4f9f-9f85-a42bb1a1fe95 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:18:48.693041 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5129ab-373c-4fe0-bc46-e479fe0951b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:18:48.693802 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5129ab-373c-4fe0-bc46-e479fe0951b7 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:18:48.694237 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5129ab-373c-4fe0-bc46-e479fe0951b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:18:48.694520 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5129ab-373c-4fe0-bc46-e479fe0951b7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:18:48.695338 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5129ab-373c-4fe0-bc46-e479fe0951b7 None admin] RBAC: Adding query filter params (role.id=d4aa3324f0c84887b41d0dc4acf7f30e, scope.project.id=76f7c0ae36ae49b09dcae4c3e9a875f4, user.id=88936b31709b429f8b6777c673ed7b8a) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:18:48.695650 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5129ab-373c-4fe0-bc46-e479fe0951b7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:18:48.697080 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5129ab-373c-4fe0-bc46-e479fe0951b7 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:18:48.702467 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 636/1260] 10.222.0.66 () {56 vars in 1374 bytes} [Mon Jul 18 02:18:48 2022] GET /identity/v3/role_assignments?user.id=88936b31709b429f8b6777c673ed7b8a&scope.project.id=76f7c0ae36ae49b09dcae4c3e9a875f4&role.id=d4aa3324f0c84887b41d0dc4acf7f30e => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:18:50.708684 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d07d56c-15a3-43bd-b1db-70c47da48900 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:18:50.709823 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d07d56c-15a3-43bd-b1db-70c47da48900 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:18:50.710150 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d07d56c-15a3-43bd-b1db-70c47da48900 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:18:50.711557 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 625/1261] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:18:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:18:50.716779 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f307dec-1c4c-41f1-884e-843e15c9b983 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:18:50.717194 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f307dec-1c4c-41f1-884e-843e15c9b983 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:18:50.717333 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f307dec-1c4c-41f1-884e-843e15c9b983 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:18:50.733923 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-5f307dec-1c4c-41f1-884e-843e15c9b983 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:18:50.735115 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-5f307dec-1c4c-41f1-884e-843e15c9b983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:18:50.800884 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 637/1262] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:18:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 02:18:50.823884 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d07d56c-15a3-43bd-b1db-70c47da48900 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:18:50.835676 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:18:50.923258 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d07d56c-15a3-43bd-b1db-70c47da48900 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:18:50.956715 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f783ee02-bd84-4568-9da8-b87594d15f44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:18:50.957495 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f783ee02-bd84-4568-9da8-b87594d15f44 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:18:50.957776 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f783ee02-bd84-4568-9da8-b87594d15f44 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:18:50.958047 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f783ee02-bd84-4568-9da8-b87594d15f44 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:18:50.968084 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f783ee02-bd84-4568-9da8-b87594d15f44 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 02:18:50.975760 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-f783ee02-bd84-4568-9da8-b87594d15f44 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 02:18:50.975760 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:18:50.975760 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:18:50.975760 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:18:50.975760 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:18:50.975760 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:18:50.975760 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:18:50.975760 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:18:50.975760 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:18:50.975760 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:18:50.975760 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:18:50.975760 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:18:50.975760 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 02:18:50.975760 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 02:18:50.975760 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 02:18:50.975760 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 02:18:50.976880 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 02:18:50.976880 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 02:18:50.976880 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 02:18:50.976880 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 02:18:50.976880 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:18:50.976880 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:18:50.976880 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:18:50.976880 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:18:50.976880 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:18:50.976880 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:18:50.976880 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:18:50.976880 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:18:50.976880 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:18:50.976880 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:18:50.976880 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:18:50.976880 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:18:50.978151 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:18:50.978151 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:18:50.978151 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:18:50.978151 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:18:50.978151 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 02:18:50.978151 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 02:18:50.978151 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 02:18:50.978151 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 02:18:50.978151 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 02:18:50.978151 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 02:18:50.978151 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 02:18:50.978151 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:18:50.978151 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 626/1263] 10.222.0.66 () {56 vars in 1096 bytes} [Mon Jul 18 02:18:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:18:50.984585 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f307dec-1c4c-41f1-884e-843e15c9b983 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:18:51.003906 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:18:51.083814 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f307dec-1c4c-41f1-884e-843e15c9b983 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:18:51.117238 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1305f3f-37e3-498a-acb7-e05811ecc92c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:18:51.117889 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1305f3f-37e3-498a-acb7-e05811ecc92c None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:18:51.118295 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1305f3f-37e3-498a-acb7-e05811ecc92c None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:18:51.118522 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1305f3f-37e3-498a-acb7-e05811ecc92c None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:18:51.119251 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1305f3f-37e3-498a-acb7-e05811ecc92c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:18:51.119552 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1305f3f-37e3-498a-acb7-e05811ecc92c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:18:51.120906 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1305f3f-37e3-498a-acb7-e05811ecc92c None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:18:51.128726 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 638/1264] 10.222.0.66 () {56 vars in 1099 bytes} [Mon Jul 18 02:18:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:18:51.135199 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f783ee02-bd84-4568-9da8-b87594d15f44 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:18:51.201722 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:18:51.234901 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f783ee02-bd84-4568-9da8-b87594d15f44 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:18:51.268232 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355d5f43-a6b3-4633-a4b6-3fad1e71f9fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:18:51.269020 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355d5f43-a6b3-4633-a4b6-3fad1e71f9fa None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:18:51.269305 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355d5f43-a6b3-4633-a4b6-3fad1e71f9fa None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:18:51.269581 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355d5f43-a6b3-4633-a4b6-3fad1e71f9fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:18:51.279842 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355d5f43-a6b3-4633-a4b6-3fad1e71f9fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:18:51.281294 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355d5f43-a6b3-4633-a4b6-3fad1e71f9fa None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:18:51.299880 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-355d5f43-a6b3-4633-a4b6-3fad1e71f9fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:18:51.299880 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:18:51.299880 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:18:51.299880 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:18:51.299880 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:18:51.299880 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:18:51.299880 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:18:51.299880 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:18:51.299880 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:18:51.299880 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:18:51.299880 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:18:51.299880 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:18:51.299880 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 02:18:51.299880 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 02:18:51.299880 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 02:18:51.300997 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 02:18:51.300997 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:18:51.300997 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:18:51.300997 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:18:51.300997 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:18:51.300997 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:18:51.300997 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:18:51.300997 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:18:51.300997 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:18:51.300997 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:18:51.300997 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:18:51.300997 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:18:51.300997 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:18:51.300997 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:18:51.300997 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:18:51.300997 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:18:51.300997 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:18:51.302180 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 02:18:51.302180 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 02:18:51.302180 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:18:51.302180 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:18:51.302180 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 627/1265] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 18 02:18:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:18:51.307576 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1305f3f-37e3-498a-acb7-e05811ecc92c None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:18:51.388657 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:18:51.406988 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1305f3f-37e3-498a-acb7-e05811ecc92c None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:18:51.440449 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bafb6f-a9e9-421e-b8b5-8b68dc459aa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:18:51.441098 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bafb6f-a9e9-421e-b8b5-8b68dc459aa3 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:18:51.441380 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bafb6f-a9e9-421e-b8b5-8b68dc459aa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:18:51.441657 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bafb6f-a9e9-421e-b8b5-8b68dc459aa3 None admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:18:51.442396 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bafb6f-a9e9-421e-b8b5-8b68dc459aa3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:18:51.442704 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bafb6f-a9e9-421e-b8b5-8b68dc459aa3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:18:51.444126 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bafb6f-a9e9-421e-b8b5-8b68dc459aa3 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:18:51.456037 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 639/1266] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 18 02:18:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:18:51.461854 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355d5f43-a6b3-4633-a4b6-3fad1e71f9fa None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:18:51.562494 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355d5f43-a6b3-4633-a4b6-3fad1e71f9fa None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:18:51.581761 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:18:51.596232 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcf7280-02b8-4058-8126-d4d737e63ca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:18:51.597036 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcf7280-02b8-4058-8126-d4d737e63ca2 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:18:51.597405 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcf7280-02b8-4058-8126-d4d737e63ca2 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:18:51.597689 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcf7280-02b8-4058-8126-d4d737e63ca2 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:18:51.606014 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcf7280-02b8-4058-8126-d4d737e63ca2 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:18:51.607515 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcf7280-02b8-4058-8126-d4d737e63ca2 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:18:51.622077 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-cfcf7280-02b8-4058-8126-d4d737e63ca2 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 18 02:18:51.622077 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:18:51.622077 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:18:51.622077 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:18:51.622077 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:18:51.622077 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:18:51.622077 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:18:51.622077 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:18:51.622077 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:18:51.622077 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:18:51.622077 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:18:51.622077 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:18:51.622077 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 02:18:51.622077 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 02:18:51.622077 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 02:18:51.622077 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 02:18:51.623246 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:18:51.623246 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:18:51.623246 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 02:18:51.623246 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 02:18:51.623246 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 02:18:51.623246 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 02:18:51.623246 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:18:51.623246 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:18:51.623246 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:18:51.623246 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:18:51.623246 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:18:51.623246 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:18:51.623246 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:18:51.623246 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:18:51.623246 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:18:51.623246 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:18:51.623246 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:18:51.623246 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:18:51.624345 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:18:51.624345 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:18:51.624345 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 02:18:51.624345 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 02:18:51.624345 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 02:18:51.624345 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 02:18:51.624345 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 02:18:51.624345 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 02:18:51.624345 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 18 02:18:51.624345 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:18:51.624345 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 628/1267] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 18 02:18:51 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:18:51.629105 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bafb6f-a9e9-421e-b8b5-8b68dc459aa3 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:18:51.730468 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bafb6f-a9e9-421e-b8b5-8b68dc459aa3 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:18:51.763429 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c4cde4-50aa-4372-bc49-90263f080aa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:18:51.764089 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c4cde4-50aa-4372-bc49-90263f080aa2 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:18:51.764375 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c4cde4-50aa-4372-bc49-90263f080aa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:18:51.764653 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c4cde4-50aa-4372-bc49-90263f080aa2 None admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:18:51.765507 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c4cde4-50aa-4372-bc49-90263f080aa2 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:18:51.765820 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c4cde4-50aa-4372-bc49-90263f080aa2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:18:51.767292 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c4cde4-50aa-4372-bc49-90263f080aa2 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:18:51.776666 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:18:51.785159 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 640/1268] 10.222.0.66 () {56 vars in 1143 bytes} [Mon Jul 18 02:18:51 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:18:51.791924 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcf7280-02b8-4058-8126-d4d737e63ca2 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:18:51.892095 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcf7280-02b8-4058-8126-d4d737e63ca2 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:18:51.925671 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c326df-4f44-4c6c-93d3-2074ee5b2cfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:18:51.926487 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c326df-4f44-4c6c-93d3-2074ee5b2cfe None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:18:51.926766 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c326df-4f44-4c6c-93d3-2074ee5b2cfe None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:18:51.927036 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c326df-4f44-4c6c-93d3-2074ee5b2cfe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:18:51.937352 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c326df-4f44-4c6c-93d3-2074ee5b2cfe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:18:51.938858 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c326df-4f44-4c6c-93d3-2074ee5b2cfe None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:18:51.955957 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-32c326df-4f44-4c6c-93d3-2074ee5b2cfe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:18:51.955957 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:18:51.955957 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:18:51.955957 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:18:51.955957 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:18:51.955957 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:18:51.955957 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:18:51.955957 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:18:51.955957 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:18:51.955957 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:18:51.955957 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:18:51.955957 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:18:51.955957 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 02:18:51.955957 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 02:18:51.955957 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 02:18:51.957112 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 02:18:51.957112 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:18:51.957112 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:18:51.957112 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:18:51.957112 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:18:51.957112 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:18:51.957112 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:18:51.957112 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:18:51.957112 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:18:51.957112 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:18:51.957112 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:18:51.957112 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:18:51.957112 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:18:51.957112 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:18:51.957112 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:18:51.957112 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:18:51.957112 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:18:51.958279 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 02:18:51.958279 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 02:18:51.958279 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:18:51.958279 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:18:51.958279 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 629/1269] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 18 02:18:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:18:51.963834 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c4cde4-50aa-4372-bc49-90263f080aa2 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:18:52.061959 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c4cde4-50aa-4372-bc49-90263f080aa2 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:18:52.096545 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3717aa4-bef5-475b-b881-3c51bcd250ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:18:52.097198 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3717aa4-bef5-475b-b881-3c51bcd250ce None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:18:52.097483 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3717aa4-bef5-475b-b881-3c51bcd250ce None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:18:52.097758 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3717aa4-bef5-475b-b881-3c51bcd250ce None admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:18:52.098506 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3717aa4-bef5-475b-b881-3c51bcd250ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:18:52.098813 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3717aa4-bef5-475b-b881-3c51bcd250ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:18:52.100279 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3717aa4-bef5-475b-b881-3c51bcd250ce None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:18:52.111418 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 641/1270] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 18 02:18:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:18:52.117545 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c326df-4f44-4c6c-93d3-2074ee5b2cfe None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:18:52.149207 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:18:52.215466 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c326df-4f44-4c6c-93d3-2074ee5b2cfe None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:18:52.248568 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a73e1d3-edc0-4ae1-8b4a-3f773136b00c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:18:52.249372 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a73e1d3-edc0-4ae1-8b4a-3f773136b00c None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:18:52.249671 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a73e1d3-edc0-4ae1-8b4a-3f773136b00c None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:18:52.249953 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a73e1d3-edc0-4ae1-8b4a-3f773136b00c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:18:52.256993 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a73e1d3-edc0-4ae1-8b4a-3f773136b00c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:18:52.258512 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a73e1d3-edc0-4ae1-8b4a-3f773136b00c None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:18:52.273731 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-0a73e1d3-edc0-4ae1-8b4a-3f773136b00c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 02:18:52.273731 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:18:52.273731 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:18:52.273731 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:18:52.273731 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:18:52.273731 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:18:52.273731 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:18:52.273731 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:18:52.273731 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:18:52.273731 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:18:52.273731 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:18:52.273731 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:18:52.273731 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 02:18:52.273731 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 02:18:52.273731 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 02:18:52.278493 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 02:18:52.278493 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:18:52.278493 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:18:52.278493 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:18:52.278493 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:18:52.278493 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:18:52.278493 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:18:52.278493 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:18:52.278493 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:18:52.278493 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:18:52.278493 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:18:52.278493 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:18:52.278493 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:18:52.278493 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:18:52.278493 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:18:52.278493 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:18:52.278493 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:18:52.279874 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 02:18:52.279874 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 02:18:52.279874 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 02:18:52.279874 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 02:18:52.279874 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 02:18:52.279874 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 02:18:52.279874 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 02:18:52.279874 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:18:52.279874 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 630/1271] 10.222.0.66 () {56 vars in 1111 bytes} [Mon Jul 18 02:18:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:18:52.280868 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3717aa4-bef5-475b-b881-3c51bcd250ce None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:18:52.339767 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:18:52.379866 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3717aa4-bef5-475b-b881-3c51bcd250ce None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:18:52.412992 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbfae80-434b-4b65-882f-05d6dd6d1ae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:18:52.413908 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbfae80-434b-4b65-882f-05d6dd6d1ae4 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:18:52.414868 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbfae80-434b-4b65-882f-05d6dd6d1ae4 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:18:52.415147 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbfae80-434b-4b65-882f-05d6dd6d1ae4 None admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:18:52.415933 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbfae80-434b-4b65-882f-05d6dd6d1ae4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:18:52.416247 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbfae80-434b-4b65-882f-05d6dd6d1ae4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:18:52.417679 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbfae80-434b-4b65-882f-05d6dd6d1ae4 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:18:52.430203 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 642/1272] 10.222.0.66 () {56 vars in 1148 bytes} [Mon Jul 18 02:18:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:18:52.436219 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a73e1d3-edc0-4ae1-8b4a-3f773136b00c None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:18:52.526917 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:18:52.535608 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a73e1d3-edc0-4ae1-8b4a-3f773136b00c None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:18:52.568895 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665ac76a-f698-4e61-9307-4158932497fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:18:52.569844 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665ac76a-f698-4e61-9307-4158932497fc None admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:18:52.570147 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665ac76a-f698-4e61-9307-4158932497fc None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:18:52.570417 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665ac76a-f698-4e61-9307-4158932497fc None admin] PATH_INFO: `/v3/projects/76f7c0ae36ae49b09dcae4c3e9a875f4/users/88936b31709b429f8b6777c673ed7b8a/roles/d4aa3324f0c84887b41d0dc4acf7f30e` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:18:52.572319 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665ac76a-f698-4e61-9307-4158932497fc None admin] RBAC: Authorizing `identity:create_grant(project_id=76f7c0ae36ae49b09dcae4c3e9a875f4, user_id=88936b31709b429f8b6777c673ed7b8a, role_id=d4aa3324f0c84887b41d0dc4acf7f30e)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:18:52.573748 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665ac76a-f698-4e61-9307-4158932497fc None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:18:52.680763 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 631/1273] 10.222.0.66 () {58 vars in 1439 bytes} [Mon Jul 18 02:18:52 2022] PUT /identity/v3/projects/76f7c0ae36ae49b09dcae4c3e9a875f4/users/88936b31709b429f8b6777c673ed7b8a/roles/d4aa3324f0c84887b41d0dc4acf7f30e => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:18:54.699856 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccda513-97fc-4a3a-98ef-0369fdc0b728 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:18:54.700382 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccda513-97fc-4a3a-98ef-0369fdc0b728 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:18:54.700499 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccda513-97fc-4a3a-98ef-0369fdc0b728 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:18:54.701802 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 643/1274] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:18:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:18:54.707062 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192b40ab-f7fa-43b8-b5b0-8f6c90c14e7b None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:18:54.707475 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192b40ab-f7fa-43b8-b5b0-8f6c90c14e7b None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:18:54.707615 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192b40ab-f7fa-43b8-b5b0-8f6c90c14e7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:18:54.725512 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-192b40ab-f7fa-43b8-b5b0-8f6c90c14e7b None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:18:54.726721 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-192b40ab-f7fa-43b8-b5b0-8f6c90c14e7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:18:54.793758 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 632/1275] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:18:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 02:18:54.816888 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccda513-97fc-4a3a-98ef-0369fdc0b728 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:18:54.920045 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccda513-97fc-4a3a-98ef-0369fdc0b728 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:18:54.937837 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:18:54.955203 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3645d00b-7df4-45f7-8cba-5a2458d98ff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:18:54.955992 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3645d00b-7df4-45f7-8cba-5a2458d98ff2 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:18:54.956278 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3645d00b-7df4-45f7-8cba-5a2458d98ff2 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:18:54.956556 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3645d00b-7df4-45f7-8cba-5a2458d98ff2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:18:54.969657 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3645d00b-7df4-45f7-8cba-5a2458d98ff2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 02:18:54.979325 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-3645d00b-7df4-45f7-8cba-5a2458d98ff2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 02:18:54.979325 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:18:54.979325 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:18:54.979325 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:18:54.979325 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:18:54.979325 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:18:54.979325 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:18:54.979325 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:18:54.979325 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:18:54.979325 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:18:54.979325 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:18:54.979325 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:18:54.979325 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 02:18:54.979325 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 02:18:54.979325 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 02:18:54.979325 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 02:18:54.980474 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 02:18:54.980474 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 02:18:54.980474 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 02:18:54.980474 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 02:18:54.980474 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:18:54.980474 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:18:54.980474 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:18:54.980474 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:18:54.980474 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:18:54.980474 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:18:54.980474 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:18:54.980474 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:18:54.980474 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:18:54.980474 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:18:54.980474 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:18:54.980474 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:18:54.981551 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:18:54.981551 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:18:54.981551 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:18:54.981551 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:18:54.981551 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 02:18:54.981551 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 02:18:54.981551 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 02:18:54.981551 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 02:18:54.981551 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 02:18:54.981551 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 02:18:54.981551 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 02:18:54.981551 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:18:54.981551 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 644/1276] 10.222.0.66 () {56 vars in 1096 bytes} [Mon Jul 18 02:18:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:18:54.985236 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192b40ab-f7fa-43b8-b5b0-8f6c90c14e7b None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:18:55.071364 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:18:55.089287 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192b40ab-f7fa-43b8-b5b0-8f6c90c14e7b None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:18:55.122782 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cad7c91-9628-4a7f-a3f8-74ca958a433c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:18:55.123459 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cad7c91-9628-4a7f-a3f8-74ca958a433c None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:18:55.123746 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cad7c91-9628-4a7f-a3f8-74ca958a433c None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:18:55.124026 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cad7c91-9628-4a7f-a3f8-74ca958a433c None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:18:55.124763 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cad7c91-9628-4a7f-a3f8-74ca958a433c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:18:55.125069 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cad7c91-9628-4a7f-a3f8-74ca958a433c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:18:55.126532 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cad7c91-9628-4a7f-a3f8-74ca958a433c None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:18:55.134956 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 633/1277] 10.222.0.66 () {56 vars in 1099 bytes} [Mon Jul 18 02:18:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:18:55.140770 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3645d00b-7df4-45f7-8cba-5a2458d98ff2 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:18:55.243093 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3645d00b-7df4-45f7-8cba-5a2458d98ff2 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:18:55.277910 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa74dce3-8993-486f-82d2-00551c028592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:18:55.278725 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa74dce3-8993-486f-82d2-00551c028592 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:18:55.279011 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa74dce3-8993-486f-82d2-00551c028592 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:18:55.279286 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa74dce3-8993-486f-82d2-00551c028592 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:18:55.289982 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa74dce3-8993-486f-82d2-00551c028592 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:18:55.292512 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa74dce3-8993-486f-82d2-00551c028592 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:18:55.308654 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-fa74dce3-8993-486f-82d2-00551c028592 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:18:55.308654 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:18:55.308654 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:18:55.308654 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:18:55.308654 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:18:55.308654 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:18:55.308654 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:18:55.308654 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:18:55.308654 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:18:55.308654 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:18:55.308654 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:18:55.308654 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:18:55.308654 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 02:18:55.308654 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 02:18:55.308654 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 02:18:55.309653 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 02:18:55.309653 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:18:55.309653 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:18:55.309653 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:18:55.309653 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:18:55.309653 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:18:55.309653 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:18:55.309653 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:18:55.309653 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:18:55.309653 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:18:55.309653 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:18:55.309653 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:18:55.309653 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:18:55.309653 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:18:55.309653 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:18:55.309653 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:18:55.309653 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:18:55.310614 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 02:18:55.310614 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 02:18:55.310614 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:18:55.310614 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:18:55.310614 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 645/1278] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 18 02:18:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:18:55.315779 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cad7c91-9628-4a7f-a3f8-74ca958a433c None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:18:55.416305 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cad7c91-9628-4a7f-a3f8-74ca958a433c None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:18:55.450286 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a9883c-7ca1-45a0-acec-d751384a8ba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:18:55.450947 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a9883c-7ca1-45a0-acec-d751384a8ba4 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:18:55.451229 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a9883c-7ca1-45a0-acec-d751384a8ba4 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:18:55.451504 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a9883c-7ca1-45a0-acec-d751384a8ba4 None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:18:55.452245 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a9883c-7ca1-45a0-acec-d751384a8ba4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:18:55.452557 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a9883c-7ca1-45a0-acec-d751384a8ba4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:18:55.453937 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a9883c-7ca1-45a0-acec-d751384a8ba4 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:18:55.466001 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 634/1279] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 18 02:18:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:18:55.471787 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa74dce3-8993-486f-82d2-00551c028592 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:18:55.502199 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:18:55.568536 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa74dce3-8993-486f-82d2-00551c028592 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:18:55.600459 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4f4f73-3ed0-46f3-80a1-f2f5f4a5a62f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:18:55.601254 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4f4f73-3ed0-46f3-80a1-f2f5f4a5a62f None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:18:55.601534 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4f4f73-3ed0-46f3-80a1-f2f5f4a5a62f None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:18:55.601802 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4f4f73-3ed0-46f3-80a1-f2f5f4a5a62f None admin] PATH_INFO: `/v3/users/placement` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:18:55.610719 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4f4f73-3ed0-46f3-80a1-f2f5f4a5a62f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:18:55.611731 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4f4f73-3ed0-46f3-80a1-f2f5f4a5a62f None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:18:55.626758 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-cf4f4f73-3ed0-46f3-80a1-f2f5f4a5a62f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 18 02:18:55.626758 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:18:55.626758 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:18:55.626758 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:18:55.626758 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:18:55.626758 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:18:55.626758 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:18:55.626758 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:18:55.626758 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:18:55.626758 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:18:55.626758 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:18:55.626758 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:18:55.626758 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 02:18:55.626758 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 02:18:55.626758 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 02:18:55.626758 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 02:18:55.627768 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:18:55.627768 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:18:55.627768 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 02:18:55.627768 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 02:18:55.627768 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 02:18:55.627768 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 02:18:55.627768 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:18:55.627768 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:18:55.627768 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:18:55.627768 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:18:55.627768 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:18:55.627768 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:18:55.627768 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:18:55.627768 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:18:55.627768 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:18:55.627768 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:18:55.627768 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:18:55.627768 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:18:55.628780 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:18:55.628780 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:18:55.628780 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 02:18:55.628780 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 02:18:55.628780 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 02:18:55.628780 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 02:18:55.628780 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 02:18:55.628780 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 02:18:55.628780 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 18 02:18:55.628780 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:18:55.628780 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 646/1280] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 18 02:18:55 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:18:55.634687 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a9883c-7ca1-45a0-acec-d751384a8ba4 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:18:55.638486 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:18:55.733352 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a9883c-7ca1-45a0-acec-d751384a8ba4 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:18:55.766467 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a724b1ce-0e67-4f6b-b9ac-8b8188c4cc8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:18:55.767084 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a724b1ce-0e67-4f6b-b9ac-8b8188c4cc8d None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:18:55.767367 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a724b1ce-0e67-4f6b-b9ac-8b8188c4cc8d None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:18:55.767642 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a724b1ce-0e67-4f6b-b9ac-8b8188c4cc8d None admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:18:55.768474 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a724b1ce-0e67-4f6b-b9ac-8b8188c4cc8d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:18:55.768785 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a724b1ce-0e67-4f6b-b9ac-8b8188c4cc8d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:18:55.770240 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a724b1ce-0e67-4f6b-b9ac-8b8188c4cc8d None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:18:55.788179 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 635/1281] 10.222.0.66 () {56 vars in 1143 bytes} [Mon Jul 18 02:18:55 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:18:55.794098 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4f4f73-3ed0-46f3-80a1-f2f5f4a5a62f None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:18:55.885754 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:18:55.894933 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4f4f73-3ed0-46f3-80a1-f2f5f4a5a62f None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:18:55.927059 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3291c6-0226-436b-bca4-e6e5244219ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:18:55.927737 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3291c6-0226-436b-bca4-e6e5244219ff None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:18:55.928016 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3291c6-0226-436b-bca4-e6e5244219ff None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:18:55.928287 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3291c6-0226-436b-bca4-e6e5244219ff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:18:55.992015 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3291c6-0226-436b-bca4-e6e5244219ff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:18:55.993452 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3291c6-0226-436b-bca4-e6e5244219ff None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:18:56.080091 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:18:56.090422 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 647/1282] 10.222.0.66 () {58 vars in 1282 bytes} [Mon Jul 18 02:18:55 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 02:18:56.096207 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a724b1ce-0e67-4f6b-b9ac-8b8188c4cc8d None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:18:56.157229 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:18:56.194746 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a724b1ce-0e67-4f6b-b9ac-8b8188c4cc8d None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:18:56.227022 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82a62fc-512d-4dd3-a01d-fa669d9b5c7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:18:56.227801 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82a62fc-512d-4dd3-a01d-fa669d9b5c7b None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:18:56.228081 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82a62fc-512d-4dd3-a01d-fa669d9b5c7b None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:18:56.228355 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82a62fc-512d-4dd3-a01d-fa669d9b5c7b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:18:56.238389 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82a62fc-512d-4dd3-a01d-fa669d9b5c7b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:18:56.240029 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82a62fc-512d-4dd3-a01d-fa669d9b5c7b None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:18:56.256455 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-f82a62fc-512d-4dd3-a01d-fa669d9b5c7b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:18:56.256455 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:18:56.256455 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:18:56.256455 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:18:56.256455 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:18:56.256455 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:18:56.256455 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:18:56.256455 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:18:56.256455 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:18:56.256455 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:18:56.256455 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:18:56.256455 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:18:56.256455 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 02:18:56.256455 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 02:18:56.256455 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 02:18:56.266768 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 02:18:56.266768 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:18:56.266768 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:18:56.266768 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:18:56.266768 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:18:56.266768 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:18:56.266768 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:18:56.266768 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:18:56.266768 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:18:56.266768 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:18:56.266768 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:18:56.266768 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:18:56.266768 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:18:56.266768 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:18:56.266768 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:18:56.266768 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:18:56.266768 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:18:56.267917 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 02:18:56.267917 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 02:18:56.267917 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 02:18:56.267917 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:18:56.267917 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 636/1283] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 18 02:18:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:18:56.268273 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3291c6-0226-436b-bca4-e6e5244219ff None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:18:56.362607 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3291c6-0226-436b-bca4-e6e5244219ff None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:18:56.397160 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7daa597-4f15-49f1-ba09-c58b9de439f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:18:56.397809 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7daa597-4f15-49f1-ba09-c58b9de439f6 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:18:56.398119 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7daa597-4f15-49f1-ba09-c58b9de439f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:18:56.398394 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7daa597-4f15-49f1-ba09-c58b9de439f6 None admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:18:56.399101 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7daa597-4f15-49f1-ba09-c58b9de439f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:18:56.399404 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7daa597-4f15-49f1-ba09-c58b9de439f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:18:56.400800 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7daa597-4f15-49f1-ba09-c58b9de439f6 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:18:56.412815 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 648/1284] 10.222.0.66 () {56 vars in 1109 bytes} [Mon Jul 18 02:18:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:18:56.418586 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82a62fc-512d-4dd3-a01d-fa669d9b5c7b None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:18:56.518275 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82a62fc-512d-4dd3-a01d-fa669d9b5c7b None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:18:56.531964 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:18:56.551808 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81553a21-5846-4292-9fa9-2b7a49857232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:18:56.552680 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81553a21-5846-4292-9fa9-2b7a49857232 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:18:56.552959 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81553a21-5846-4292-9fa9-2b7a49857232 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:18:56.553237 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81553a21-5846-4292-9fa9-2b7a49857232 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:18:56.559776 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81553a21-5846-4292-9fa9-2b7a49857232 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:18:56.561235 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81553a21-5846-4292-9fa9-2b7a49857232 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:18:56.576666 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-81553a21-5846-4292-9fa9-2b7a49857232 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 02:18:56.576666 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:18:56.576666 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:18:56.576666 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:18:56.576666 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:18:56.576666 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:18:56.576666 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:18:56.576666 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:18:56.576666 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:18:56.576666 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:18:56.576666 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:18:56.576666 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:18:56.576666 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 02:18:56.576666 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 02:18:56.576666 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 02:18:56.577786 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 02:18:56.577786 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:18:56.577786 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:18:56.577786 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:18:56.577786 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:18:56.577786 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:18:56.577786 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:18:56.577786 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:18:56.577786 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jul 18 02:18:56.577786 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:18:56.577786 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:18:56.577786 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:18:56.577786 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:18:56.577786 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:18:56.577786 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:18:56.577786 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:18:56.577786 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:18:56.578957 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 02:18:56.578957 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 02:18:56.578957 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 02:18:56.578957 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 02:18:56.578957 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 02:18:56.578957 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 02:18:56.578957 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 02:18:56.578957 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jul 18 02:18:56.578957 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 637/1285] 10.222.0.66 () {56 vars in 1111 bytes} [Mon Jul 18 02:18:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:18:56.584473 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7daa597-4f15-49f1-ba09-c58b9de439f6 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:18:56.636105 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:18:56.686002 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7daa597-4f15-49f1-ba09-c58b9de439f6 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:18:56.720513 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d938bd68-3981-4bd2-9df9-79aa612e227e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:18:56.721166 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d938bd68-3981-4bd2-9df9-79aa612e227e None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:18:56.721446 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d938bd68-3981-4bd2-9df9-79aa612e227e None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:18:56.721719 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d938bd68-3981-4bd2-9df9-79aa612e227e None admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:18:56.722526 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d938bd68-3981-4bd2-9df9-79aa612e227e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:18:56.722830 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d938bd68-3981-4bd2-9df9-79aa612e227e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:18:56.724398 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d938bd68-3981-4bd2-9df9-79aa612e227e None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:18:56.736139 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 649/1286] 10.222.0.66 () {56 vars in 1148 bytes} [Mon Jul 18 02:18:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:18:56.742416 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81553a21-5846-4292-9fa9-2b7a49857232 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:18:56.846526 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81553a21-5846-4292-9fa9-2b7a49857232 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:18:56.879768 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f60fae-afed-4a49-b104-d2a4c8955587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:18:56.881133 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f60fae-afed-4a49-b104-d2a4c8955587 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:18:56.881459 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f60fae-afed-4a49-b104-d2a4c8955587 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:18:56.881727 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f60fae-afed-4a49-b104-d2a4c8955587 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:18:56.882574 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f60fae-afed-4a49-b104-d2a4c8955587 None admin] RBAC: Adding query filter params (role.id=d4aa3324f0c84887b41d0dc4acf7f30e, scope.project.id=76f7c0ae36ae49b09dcae4c3e9a875f4, user.id=88936b31709b429f8b6777c673ed7b8a) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:18:56.886948 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f60fae-afed-4a49-b104-d2a4c8955587 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:18:56.886948 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f60fae-afed-4a49-b104-d2a4c8955587 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:18:56.889985 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 638/1287] 10.222.0.66 () {56 vars in 1374 bytes} [Mon Jul 18 02:18:56 2022] GET /identity/v3/role_assignments?user.id=88936b31709b429f8b6777c673ed7b8a&scope.project.id=76f7c0ae36ae49b09dcae4c3e9a875f4&role.id=d4aa3324f0c84887b41d0dc4acf7f30e => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:18:58.912802 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc550d3b-4f93-4c56-ad78-6f7aa5a00f95 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:18:58.913325 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc550d3b-4f93-4c56-ad78-6f7aa5a00f95 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:18:58.913325 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc550d3b-4f93-4c56-ad78-6f7aa5a00f95 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:18:58.914756 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 650/1288] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:18:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 02:18:58.920144 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7dfe0c-edae-4fbc-bffd-c4c945045d38 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:18:58.920583 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7dfe0c-edae-4fbc-bffd-c4c945045d38 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:18:58.920704 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7dfe0c-edae-4fbc-bffd-c4c945045d38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:18:58.937102 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-db7dfe0c-edae-4fbc-bffd-c4c945045d38 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:18:58.938309 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-db7dfe0c-edae-4fbc-bffd-c4c945045d38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:18:58.968152 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:18:59.002935 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 639/1289] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:18:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 02:18:59.025876 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc550d3b-4f93-4c56-ad78-6f7aa5a00f95 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:18:59.124926 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc550d3b-4f93-4c56-ad78-6f7aa5a00f95 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:18:59.136663 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:18:59.159121 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38377f23-b37d-45b1-82b8-afc16fe95757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:18:59.159898 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38377f23-b37d-45b1-82b8-afc16fe95757 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:18:59.160182 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38377f23-b37d-45b1-82b8-afc16fe95757 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:18:59.160509 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38377f23-b37d-45b1-82b8-afc16fe95757 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:18:59.161367 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38377f23-b37d-45b1-82b8-afc16fe95757 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:18:59.163302 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38377f23-b37d-45b1-82b8-afc16fe95757 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:18:59.174744 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-38377f23-b37d-45b1-82b8-afc16fe95757 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 18 02:18:59.174744 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:18:59.174744 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 02:18:59.174744 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 02:18:59.174744 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 02:18:59.174744 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 02:18:59.174744 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 02:18:59.174744 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 02:18:59.174744 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 18 02:18:59.174744 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:18:59.174744 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 02:18:59.174744 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:18:59.174744 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:18:59.174744 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:18:59.174744 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:18:59.174744 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:18:59.174744 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:18:59.178191 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:18:59.178191 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:18:59.178191 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:18:59.178191 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:18:59.178191 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:18:59.178191 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:18:59.178191 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 02:18:59.178191 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 02:18:59.178191 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 02:18:59.178191 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 02:18:59.178191 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:18:59.178191 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:18:59.178191 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:18:59.178191 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:18:59.178191 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:18:59.178191 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:18:59.179245 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:18:59.179245 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:18:59.179245 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:18:59.179245 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:18:59.179245 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:18:59.179245 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:18:59.179245 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:18:59.179245 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:18:59.179245 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:18:59.179245 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:18:59.179245 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 02:18:59.179245 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 02:18:59.179245 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 18 02:18:59.179245 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:18:59.179245 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 651/1290] 10.222.0.66 () {56 vars in 1117 bytes} [Mon Jul 18 02:18:59 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:18:59.182653 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7dfe0c-edae-4fbc-bffd-c4c945045d38 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:18:59.280090 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7dfe0c-edae-4fbc-bffd-c4c945045d38 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:18:59.313176 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b365b7f-c1f6-4ade-8232-cdcdc11ec135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:18:59.313832 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b365b7f-c1f6-4ade-8232-cdcdc11ec135 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:18:59.314137 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b365b7f-c1f6-4ade-8232-cdcdc11ec135 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:18:59.314415 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b365b7f-c1f6-4ade-8232-cdcdc11ec135 None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:18:59.315111 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b365b7f-c1f6-4ade-8232-cdcdc11ec135 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:18:59.315416 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b365b7f-c1f6-4ade-8232-cdcdc11ec135 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:18:59.316790 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b365b7f-c1f6-4ade-8232-cdcdc11ec135 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:18:59.322116 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 640/1291] 10.222.0.66 () {56 vars in 1116 bytes} [Mon Jul 18 02:18:59 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:18:59.328080 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38377f23-b37d-45b1-82b8-afc16fe95757 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:18:59.426020 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38377f23-b37d-45b1-82b8-afc16fe95757 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:18:59.460265 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ca6b9e-b1f3-4cba-b20e-4946417ed5c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:18:59.460939 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ca6b9e-b1f3-4cba-b20e-4946417ed5c4 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:18:59.461224 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ca6b9e-b1f3-4cba-b20e-4946417ed5c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:18:59.461502 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ca6b9e-b1f3-4cba-b20e-4946417ed5c4 None admin] PATH_INFO: `/v3/services` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:18:59.462259 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ca6b9e-b1f3-4cba-b20e-4946417ed5c4 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:18:59.462567 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ca6b9e-b1f3-4cba-b20e-4946417ed5c4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:18:59.463934 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ca6b9e-b1f3-4cba-b20e-4946417ed5c4 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:18:59.468427 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 652/1292] 10.222.0.66 () {56 vars in 1116 bytes} [Mon Jul 18 02:18:59 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:19:01.468678 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e887976-1819-4c73-b4fc-4e79e167c1a8 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:19:01.469229 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e887976-1819-4c73-b4fc-4e79e167c1a8 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:19:01.469229 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e887976-1819-4c73-b4fc-4e79e167c1a8 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:19:01.470658 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 641/1293] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:19:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:19:01.475942 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0130b64-9094-48a0-8b83-ce772449f911 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:19:01.476405 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0130b64-9094-48a0-8b83-ce772449f911 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:19:01.476538 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0130b64-9094-48a0-8b83-ce772449f911 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:19:01.494872 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-a0130b64-9094-48a0-8b83-ce772449f911 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:19:01.496068 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-a0130b64-9094-48a0-8b83-ce772449f911 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:19:01.526734 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:19:01.563044 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 653/1294] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:19:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 02:19:01.585882 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e887976-1819-4c73-b4fc-4e79e167c1a8 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:19:01.616742 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:19:01.686190 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e887976-1819-4c73-b4fc-4e79e167c1a8 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:19:01.719002 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d729a3f-6140-48a6-a778-6be1375b786e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:19:01.719594 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d729a3f-6140-48a6-a778-6be1375b786e None admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:19:01.719872 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d729a3f-6140-48a6-a778-6be1375b786e None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:19:01.720144 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d729a3f-6140-48a6-a778-6be1375b786e None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:19:01.720847 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d729a3f-6140-48a6-a778-6be1375b786e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:19:01.722233 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d729a3f-6140-48a6-a778-6be1375b786e None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:19:01.833843 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 642/1295] 10.222.0.66 () {60 vars in 1141 bytes} [Mon Jul 18 02:19:01 2022] POST /identity/v3/services => generated 246 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:19:03.834662 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d297b73c-943a-4bf8-a88b-e161c3198544 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:19:03.835203 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d297b73c-943a-4bf8-a88b-e161c3198544 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:19:03.835203 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d297b73c-943a-4bf8-a88b-e161c3198544 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:19:03.836815 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 654/1296] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:19:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:19:03.841997 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f0d181-a186-45c5-8b10-db02be8418f5 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:19:03.844824 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f0d181-a186-45c5-8b10-db02be8418f5 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:19:03.844824 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f0d181-a186-45c5-8b10-db02be8418f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:19:03.858907 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-c8f0d181-a186-45c5-8b10-db02be8418f5 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:19:03.860129 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-c8f0d181-a186-45c5-8b10-db02be8418f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:19:03.931278 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 643/1297] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:19:03 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 02:19:03.955041 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d297b73c-943a-4bf8-a88b-e161c3198544 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:19:04.056594 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d297b73c-943a-4bf8-a88b-e161c3198544 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:19:04.092090 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebc250f-6fd2-49a1-b544-57451dad3742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:19:04.092875 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebc250f-6fd2-49a1-b544-57451dad3742 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:19:04.093162 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebc250f-6fd2-49a1-b544-57451dad3742 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:19:04.093442 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebc250f-6fd2-49a1-b544-57451dad3742 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:19:04.094181 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebc250f-6fd2-49a1-b544-57451dad3742 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:19:04.095558 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebc250f-6fd2-49a1-b544-57451dad3742 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:19:04.110776 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-9ebc250f-6fd2-49a1-b544-57451dad3742 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 18 02:19:04.110776 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:19:04.110776 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 02:19:04.110776 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 02:19:04.110776 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 02:19:04.110776 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 02:19:04.110776 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 02:19:04.110776 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 02:19:04.110776 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 18 02:19:04.110776 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:19:04.110776 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 02:19:04.110776 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:19:04.110776 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:19:04.110776 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:19:04.110776 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:19:04.110776 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:19:04.110776 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:19:04.111836 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:19:04.111836 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:19:04.111836 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:19:04.111836 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:19:04.111836 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:19:04.111836 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:19:04.111836 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 02:19:04.111836 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 02:19:04.111836 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 02:19:04.111836 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 02:19:04.111836 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:19:04.111836 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:19:04.111836 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:19:04.111836 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:19:04.111836 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:19:04.111836 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:19:04.112764 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:19:04.112764 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:19:04.112764 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:19:04.112764 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:19:04.112764 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:19:04.112764 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:19:04.112764 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:19:04.112764 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:19:04.112764 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:19:04.112764 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:19:04.112764 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 02:19:04.112764 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 02:19:04.112764 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 18 02:19:04.112764 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:19:04.112764 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 655/1298] 10.222.0.66 () {56 vars in 1117 bytes} [Mon Jul 18 02:19:03 2022] GET /identity/v3/services/placement => generated 90 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:19:04.118547 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f0d181-a186-45c5-8b10-db02be8418f5 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:19:04.137192 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:19:04.217621 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f0d181-a186-45c5-8b10-db02be8418f5 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:19:04.252211 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9b0cca-d290-4ed2-899f-fac0da7a841d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:19:04.252867 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9b0cca-d290-4ed2-899f-fac0da7a841d None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:19:04.253151 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9b0cca-d290-4ed2-899f-fac0da7a841d None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:19:04.253430 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9b0cca-d290-4ed2-899f-fac0da7a841d None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:19:04.254653 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9b0cca-d290-4ed2-899f-fac0da7a841d None admin] RBAC: Adding query filter params (name=placement) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:19:04.254964 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9b0cca-d290-4ed2-899f-fac0da7a841d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:19:04.256390 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9b0cca-d290-4ed2-899f-fac0da7a841d None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:19:04.261470 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 644/1299] 10.222.0.66 () {56 vars in 1116 bytes} [Mon Jul 18 02:19:04 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:19:04.267581 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebc250f-6fd2-49a1-b544-57451dad3742 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:19:04.299610 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:19:04.369800 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebc250f-6fd2-49a1-b544-57451dad3742 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:19:04.403354 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5213f1a5-6fc0-4bbe-8696-6c9523acac69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:19:04.404004 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5213f1a5-6fc0-4bbe-8696-6c9523acac69 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:19:04.404289 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5213f1a5-6fc0-4bbe-8696-6c9523acac69 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:19:04.404566 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5213f1a5-6fc0-4bbe-8696-6c9523acac69 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:19:04.405376 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5213f1a5-6fc0-4bbe-8696-6c9523acac69 None admin] RBAC: Adding query filter params (interface=public, service_id=d7b333fa4d3f40b4a2a90fa0d849ff76, region_id=RegionOne) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:19:04.405693 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5213f1a5-6fc0-4bbe-8696-6c9523acac69 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:19:04.407096 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5213f1a5-6fc0-4bbe-8696-6c9523acac69 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:19:04.412530 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 656/1300] 10.222.0.66 () {56 vars in 1251 bytes} [Mon Jul 18 02:19:04 2022] GET /identity/v3/endpoints?service_id=d7b333fa4d3f40b4a2a90fa0d849ff76&interface=public®ion_id=RegionOne => generated 194 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:19:04.418232 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9b0cca-d290-4ed2-899f-fac0da7a841d None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:19:04.497789 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:19:04.515903 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9b0cca-d290-4ed2-899f-fac0da7a841d None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:19:04.548698 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf0f287-4a76-4aab-9ecc-a5e1a962a6b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:19:04.549368 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf0f287-4a76-4aab-9ecc-a5e1a962a6b8 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:19:04.549653 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf0f287-4a76-4aab-9ecc-a5e1a962a6b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:19:04.549928 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf0f287-4a76-4aab-9ecc-a5e1a962a6b8 None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:19:04.550677 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf0f287-4a76-4aab-9ecc-a5e1a962a6b8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:19:04.552626 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf0f287-4a76-4aab-9ecc-a5e1a962a6b8 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:19:04.559522 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 645/1301] 10.222.0.66 () {56 vars in 1087 bytes} [Mon Jul 18 02:19:04 2022] GET /identity/v3/services => generated 2191 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:19:06.547366 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5356ee7-c9e3-486d-bd2e-69c2d0776af1 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:19:06.547917 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5356ee7-c9e3-486d-bd2e-69c2d0776af1 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:19:06.547917 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5356ee7-c9e3-486d-bd2e-69c2d0776af1 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:19:06.549282 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 657/1302] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:19:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:19:06.554540 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83cb725-56ce-4f0f-8c5f-022b14649bdd None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:19:06.554955 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83cb725-56ce-4f0f-8c5f-022b14649bdd None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:19:06.555108 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83cb725-56ce-4f0f-8c5f-022b14649bdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:19:06.572372 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-e83cb725-56ce-4f0f-8c5f-022b14649bdd None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:19:06.573648 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-e83cb725-56ce-4f0f-8c5f-022b14649bdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:19:06.640769 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 646/1303] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:19:06 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 02:19:06.662914 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5356ee7-c9e3-486d-bd2e-69c2d0776af1 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:19:06.757986 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:19:06.764444 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5356ee7-c9e3-486d-bd2e-69c2d0776af1 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:19:06.796863 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c08db6-a4d3-4187-a315-c372c58675df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:19:06.798172 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c08db6-a4d3-4187-a315-c372c58675df None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:19:06.798172 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c08db6-a4d3-4187-a315-c372c58675df None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:19:06.798172 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c08db6-a4d3-4187-a315-c372c58675df None admin] PATH_INFO: `/v3/services/placement` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:19:06.798757 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c08db6-a4d3-4187-a315-c372c58675df None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:19:06.800114 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c08db6-a4d3-4187-a315-c372c58675df None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:19:06.812438 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-b7c08db6-a4d3-4187-a315-c372c58675df None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 18 02:19:06.812438 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:19:06.812438 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 02:19:06.812438 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 02:19:06.812438 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 02:19:06.812438 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 02:19:06.812438 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 02:19:06.812438 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 02:19:06.812438 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 18 02:19:06.812438 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:19:06.812438 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 02:19:06.812438 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:19:06.812438 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 02:19:06.812438 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 02:19:06.812438 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 02:19:06.812438 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 02:19:06.812438 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 02:19:06.813608 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 02:19:06.813608 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 02:19:06.813608 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 02:19:06.813608 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 02:19:06.813608 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 02:19:06.813608 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 02:19:06.813608 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 02:19:06.813608 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 02:19:06.813608 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 02:19:06.813608 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 02:19:06.813608 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 02:19:06.813608 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 02:19:06.813608 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 02:19:06.813608 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 02:19:06.813608 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 02:19:06.813608 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 02:19:06.814563 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 02:19:06.814563 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 18 02:19:06.814563 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 02:19:06.814563 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 18 02:19:06.814563 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 02:19:06.814563 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 02:19:06.814563 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 02:19:06.814563 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 18 02:19:06.814563 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 02:19:06.814563 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 18 02:19:06.814563 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 02:19:06.814563 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 02:19:06.814563 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 18 02:19:06.814563 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 18 02:19:06.814563 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 658/1304] 10.222.0.66 () {56 vars in 1117 bytes} [Mon Jul 18 02:19:06 2022] GET /identity/v3/services/placement => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 02:19:06.820594 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83cb725-56ce-4f0f-8c5f-022b14649bdd None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:19:06.894615 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:19:06.916383 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83cb725-56ce-4f0f-8c5f-022b14649bdd None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:19:06.948929 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bf278b-db8f-477c-9b2a-d5175ea695c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:19:06.949578 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bf278b-db8f-477c-9b2a-d5175ea695c1 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:19:06.949859 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bf278b-db8f-477c-9b2a-d5175ea695c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:19:06.950161 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bf278b-db8f-477c-9b2a-d5175ea695c1 None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:19:06.950852 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bf278b-db8f-477c-9b2a-d5175ea695c1 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:19:06.951154 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bf278b-db8f-477c-9b2a-d5175ea695c1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:19:06.952510 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bf278b-db8f-477c-9b2a-d5175ea695c1 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:19:06.957581 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 647/1305] 10.222.0.66 () {56 vars in 1116 bytes} [Mon Jul 18 02:19:06 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:19:06.963599 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c08db6-a4d3-4187-a315-c372c58675df None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:19:07.062976 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c08db6-a4d3-4187-a315-c372c58675df None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:19:07.095361 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd619db-e35e-4555-ab6d-d7f9d48a0b0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:19:07.096007 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd619db-e35e-4555-ab6d-d7f9d48a0b0a None admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:19:07.096291 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd619db-e35e-4555-ab6d-d7f9d48a0b0a None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:19:07.096564 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd619db-e35e-4555-ab6d-d7f9d48a0b0a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:19:07.097306 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd619db-e35e-4555-ab6d-d7f9d48a0b0a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:19:07.099360 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd619db-e35e-4555-ab6d-d7f9d48a0b0a None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:19:07.192401 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 659/1306] 10.222.0.66 () {60 vars in 1144 bytes} [Mon Jul 18 02:19:06 2022] POST /identity/v3/endpoints => generated 331 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:19:16.963910 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=30 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:19:17.187667 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:26:31.868303 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d95d358-d6e5-4a41-ac00-9770aea71f42 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:26:31.869421 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d95d358-d6e5-4a41-ac00-9770aea71f42 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:26:31.869920 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d95d358-d6e5-4a41-ac00-9770aea71f42 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:26:31.878170 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb8d67d-ce60-435f-ac9b-0621f72b0913 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:26:31.879185 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb8d67d-ce60-435f-ac9b-0621f72b0913 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:26:31.879673 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb8d67d-ce60-435f-ac9b-0621f72b0913 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:26:31.883535 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 648/1307] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:26:31 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:26:31.973480 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-9eb8d67d-ce60-435f-ac9b-0621f72b0913 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:26:31.975280 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-9eb8d67d-ce60-435f-ac9b-0621f72b0913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:26:32.101465 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 660/1308] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:26:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 02:26:32.155165 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2183e5c7-7275-4cee-82d7-34da83b387c8 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:26:32.155791 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2183e5c7-7275-4cee-82d7-34da83b387c8 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:26:32.156263 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2183e5c7-7275-4cee-82d7-34da83b387c8 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:26:32.158460 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 649/1309] 10.222.0.66 () {54 vars in 965 bytes} [Mon Jul 18 02:26:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:26:32.164748 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92782976-0f85-4bd5-a3bb-25861d560431 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:26:32.165125 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92782976-0f85-4bd5-a3bb-25861d560431 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:26:32.165464 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92782976-0f85-4bd5-a3bb-25861d560431 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:26:32.277385 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-92782976-0f85-4bd5-a3bb-25861d560431 None None] MFA Rules not processed for user `eaf19a259c2741cea375a38763e33cdc`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:26:32.279310 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-92782976-0f85-4bd5-a3bb-25861d560431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:26:32.334260 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:26:32.410042 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 661/1310] 10.222.0.66 () {58 vars in 1062 bytes} [Mon Jul 18 02:26:32 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:26:32.429515 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2183e5c7-7275-4cee-82d7-34da83b387c8 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:26:32.431332 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-2183e5c7-7275-4cee-82d7-34da83b387c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:26:32.549126 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2183e5c7-7275-4cee-82d7-34da83b387c8 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:26:32.636643 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c750062f-c1a6-4881-92b3-5e096cec1230 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eaf19a259c2741cea375a38763e33cdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:26:32.637251 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c750062f-c1a6-4881-92b3-5e096cec1230 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:26:32.637539 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c750062f-c1a6-4881-92b3-5e096cec1230 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:26:32.637817 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c750062f-c1a6-4881-92b3-5e096cec1230 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:26:32.639487 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-c750062f-c1a6-4881-92b3-5e096cec1230 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:26:32.704899 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:26:32.769982 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c750062f-c1a6-4881-92b3-5e096cec1230 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:26:32.771492 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c750062f-c1a6-4881-92b3-5e096cec1230 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:26:32.792849 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 650/1311] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:26:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:26:35.132733 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5396ab6a-f1e9-4615-a954-16ac0cb207e0 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:26:35.133320 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5396ab6a-f1e9-4615-a954-16ac0cb207e0 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:26:35.133578 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5396ab6a-f1e9-4615-a954-16ac0cb207e0 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:26:35.135123 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 662/1312] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:26:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:26:35.141607 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677b56df-adc5-4e82-a839-96c2356fc30e None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:26:35.142149 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677b56df-adc5-4e82-a839-96c2356fc30e None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:26:35.142363 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677b56df-adc5-4e82-a839-96c2356fc30e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:26:35.224760 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-677b56df-adc5-4e82-a839-96c2356fc30e None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:26:35.226018 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-677b56df-adc5-4e82-a839-96c2356fc30e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:26:35.328626 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 651/1313] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:26:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:26:35.366837 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5396ab6a-f1e9-4615-a954-16ac0cb207e0 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:26:35.368381 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-5396ab6a-f1e9-4615-a954-16ac0cb207e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:26:35.495764 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5396ab6a-f1e9-4615-a954-16ac0cb207e0 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:26:35.585468 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:26:35.588114 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f180df4-43bf-44cf-a2ba-7b33d35a7b0c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eaf19a259c2741cea375a38763e33cdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:26:35.588769 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f180df4-43bf-44cf-a2ba-7b33d35a7b0c service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:26:35.589049 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f180df4-43bf-44cf-a2ba-7b33d35a7b0c service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:26:35.589321 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f180df4-43bf-44cf-a2ba-7b33d35a7b0c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:26:35.591851 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-7f180df4-43bf-44cf-a2ba-7b33d35a7b0c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:26:35.705719 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f180df4-43bf-44cf-a2ba-7b33d35a7b0c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:26:35.707199 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f180df4-43bf-44cf-a2ba-7b33d35a7b0c service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:26:35.725889 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 663/1314] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:26:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:26:38.258860 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37d985a-cb3d-4d81-93f0-646eb9e4b431 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:26:38.259410 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37d985a-cb3d-4d81-93f0-646eb9e4b431 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:26:38.259410 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37d985a-cb3d-4d81-93f0-646eb9e4b431 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:26:38.260837 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 652/1315] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:26:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:26:38.266169 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc1d47e-6cb5-43df-a12e-5a8542f64dec None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:26:38.266615 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc1d47e-6cb5-43df-a12e-5a8542f64dec None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:26:38.266713 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc1d47e-6cb5-43df-a12e-5a8542f64dec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:26:38.330578 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-cfc1d47e-6cb5-43df-a12e-5a8542f64dec None None] MFA Rules not processed for user `8fa31761548948049aa25bd5881382c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:26:38.331788 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-cfc1d47e-6cb5-43df-a12e-5a8542f64dec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:26:38.382975 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:26:38.427208 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 664/1316] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:26:38 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:26:38.467743 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37d985a-cb3d-4d81-93f0-646eb9e4b431 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:26:38.469277 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-f37d985a-cb3d-4d81-93f0-646eb9e4b431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:26:38.479257 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:26:38.571699 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37d985a-cb3d-4d81-93f0-646eb9e4b431 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:26:38.644447 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33a0f98-5c09-4aa7-97ff-b03605952e8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eaf19a259c2741cea375a38763e33cdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:26:38.645094 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33a0f98-5c09-4aa7-97ff-b03605952e8f service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:26:38.645373 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33a0f98-5c09-4aa7-97ff-b03605952e8f service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:26:38.645644 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33a0f98-5c09-4aa7-97ff-b03605952e8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:26:38.647301 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-c33a0f98-5c09-4aa7-97ff-b03605952e8f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:26:38.709435 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:26:38.755601 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33a0f98-5c09-4aa7-97ff-b03605952e8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:26:38.757071 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33a0f98-5c09-4aa7-97ff-b03605952e8f service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:26:38.777039 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 653/1317] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:26:38 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:26:41.782674 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bb6d4d-3969-41ad-b9f0-d5ab18c5fcb4 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:26:41.783194 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bb6d4d-3969-41ad-b9f0-d5ab18c5fcb4 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:26:41.783194 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bb6d4d-3969-41ad-b9f0-d5ab18c5fcb4 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:26:41.784642 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 665/1318] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:26:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:26:41.789724 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e72d8c-4b1a-4451-9d55-e2859f4b1210 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:26:41.790128 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e72d8c-4b1a-4451-9d55-e2859f4b1210 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:26:41.790320 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e72d8c-4b1a-4451-9d55-e2859f4b1210 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:26:41.853839 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-16e72d8c-4b1a-4451-9d55-e2859f4b1210 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:26:41.855094 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-16e72d8c-4b1a-4451-9d55-e2859f4b1210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:26:41.948658 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 654/1319] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:26:41 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:26:41.988163 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bb6d4d-3969-41ad-b9f0-d5ab18c5fcb4 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:26:41.989678 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-94bb6d4d-3969-41ad-b9f0-d5ab18c5fcb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:26:42.092115 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bb6d4d-3969-41ad-b9f0-d5ab18c5fcb4 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:26:42.162414 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484e6d5e-f50c-4e88-a99a-eecd7a49be79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eaf19a259c2741cea375a38763e33cdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:26:42.163050 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484e6d5e-f50c-4e88-a99a-eecd7a49be79 service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:26:42.163327 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484e6d5e-f50c-4e88-a99a-eecd7a49be79 service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:26:42.163603 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484e6d5e-f50c-4e88-a99a-eecd7a49be79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:26:42.165214 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-484e6d5e-f50c-4e88-a99a-eecd7a49be79 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:26:42.173416 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:26:42.272097 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484e6d5e-f50c-4e88-a99a-eecd7a49be79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:26:42.273533 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484e6d5e-f50c-4e88-a99a-eecd7a49be79 service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:26:42.291910 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 666/1320] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:26:41 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:26:45.355818 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c60c44-2a07-4a2e-9df4-c059416cedff None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:26:45.356376 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c60c44-2a07-4a2e-9df4-c059416cedff None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:26:45.356376 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c60c44-2a07-4a2e-9df4-c059416cedff None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:26:45.357734 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 655/1321] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:26:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:26:45.362935 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f7e51c-54e5-46f5-b90b-024d2383953e None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:26:45.363360 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f7e51c-54e5-46f5-b90b-024d2383953e None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:26:45.363502 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f7e51c-54e5-46f5-b90b-024d2383953e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:26:45.429382 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-99f7e51c-54e5-46f5-b90b-024d2383953e None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:26:45.430645 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-99f7e51c-54e5-46f5-b90b-024d2383953e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:26:45.498619 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:26:45.538408 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 667/1322] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:26:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:26:45.576055 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c60c44-2a07-4a2e-9df4-c059416cedff None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:26:45.577595 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-74c60c44-2a07-4a2e-9df4-c059416cedff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:26:45.599298 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:26:45.694788 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c60c44-2a07-4a2e-9df4-c059416cedff None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:26:45.772318 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae99e6c9-b8a2-4069-ae1b-154d8b0af868 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eaf19a259c2741cea375a38763e33cdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:26:45.772980 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae99e6c9-b8a2-4069-ae1b-154d8b0af868 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:26:45.773265 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae99e6c9-b8a2-4069-ae1b-154d8b0af868 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:26:45.773540 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae99e6c9-b8a2-4069-ae1b-154d8b0af868 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:26:45.775336 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-ae99e6c9-b8a2-4069-ae1b-154d8b0af868 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:26:45.847593 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:26:45.885888 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae99e6c9-b8a2-4069-ae1b-154d8b0af868 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:26:45.887380 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae99e6c9-b8a2-4069-ae1b-154d8b0af868 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:26:45.907663 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 656/1323] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:26:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 02:26:48.015972 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd1e3b7-7802-43e6-8c46-c7b5ba14f5fb None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:26:48.016536 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd1e3b7-7802-43e6-8c46-c7b5ba14f5fb None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:26:48.016536 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd1e3b7-7802-43e6-8c46-c7b5ba14f5fb None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:26:48.017921 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 668/1324] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:26:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:26:48.023164 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d8a69b-4bbb-46fe-b9da-dc8b0e3cde65 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:26:48.023636 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d8a69b-4bbb-46fe-b9da-dc8b0e3cde65 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:26:48.023843 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d8a69b-4bbb-46fe-b9da-dc8b0e3cde65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:26:48.088806 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-75d8a69b-4bbb-46fe-b9da-dc8b0e3cde65 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:26:48.090205 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-75d8a69b-4bbb-46fe-b9da-dc8b0e3cde65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:26:48.186973 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 657/1325] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:26:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:26:48.224515 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd1e3b7-7802-43e6-8c46-c7b5ba14f5fb None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:26:48.226138 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-6cd1e3b7-7802-43e6-8c46-c7b5ba14f5fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:26:48.331260 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd1e3b7-7802-43e6-8c46-c7b5ba14f5fb None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:26:48.404778 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7545e6-8ef8-46cb-99a8-b82f63f0ac6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eaf19a259c2741cea375a38763e33cdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:26:48.405429 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7545e6-8ef8-46cb-99a8-b82f63f0ac6e service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:26:48.405717 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7545e6-8ef8-46cb-99a8-b82f63f0ac6e service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:26:48.405994 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7545e6-8ef8-46cb-99a8-b82f63f0ac6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:26:48.407670 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-ff7545e6-8ef8-46cb-99a8-b82f63f0ac6e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:26:48.425191 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:26:48.518585 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7545e6-8ef8-46cb-99a8-b82f63f0ac6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:26:48.520048 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7545e6-8ef8-46cb-99a8-b82f63f0ac6e service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:26:48.538659 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 669/1326] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:26:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:26:50.640816 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32abf98-17ce-44e4-b8bb-0e25b8035a0e None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:26:50.641350 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32abf98-17ce-44e4-b8bb-0e25b8035a0e None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:26:50.641350 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32abf98-17ce-44e4-b8bb-0e25b8035a0e None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:26:50.642804 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 658/1327] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:26:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:26:50.647918 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaea9a2-8904-4261-aea4-b70cdad53da6 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:26:50.648326 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaea9a2-8904-4261-aea4-b70cdad53da6 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:26:50.648482 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaea9a2-8904-4261-aea4-b70cdad53da6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:26:50.722697 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-abaea9a2-8904-4261-aea4-b70cdad53da6 None None] MFA Rules not processed for user `8fa31761548948049aa25bd5881382c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:26:50.723915 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-abaea9a2-8904-4261-aea4-b70cdad53da6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:26:50.787213 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:26:50.818817 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 670/1328] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:26:50 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:26:50.876194 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05242e4-433a-4a47-b29f-6720103a1d6a None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:26:50.876480 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05242e4-433a-4a47-b29f-6720103a1d6a None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:26:50.876844 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05242e4-433a-4a47-b29f-6720103a1d6a None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:26:50.878335 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 659/1329] 10.222.0.66 () {54 vars in 965 bytes} [Mon Jul 18 02:26:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:26:50.884764 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b1f522-6e56-4d5f-acaf-5b11bbdc9ac8 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:26:50.885056 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b1f522-6e56-4d5f-acaf-5b11bbdc9ac8 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:26:50.885336 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b1f522-6e56-4d5f-acaf-5b11bbdc9ac8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:26:50.961540 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-36b1f522-6e56-4d5f-acaf-5b11bbdc9ac8 None None] MFA Rules not processed for user `eaf19a259c2741cea375a38763e33cdc`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:26:50.962816 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-36b1f522-6e56-4d5f-acaf-5b11bbdc9ac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:26:51.045703 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 671/1330] 10.222.0.66 () {58 vars in 1062 bytes} [Mon Jul 18 02:26:50 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:26:51.054653 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05242e4-433a-4a47-b29f-6720103a1d6a None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:26:51.056168 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-d05242e4-433a-4a47-b29f-6720103a1d6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:26:51.087489 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:26:51.162200 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05242e4-433a-4a47-b29f-6720103a1d6a None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:26:51.237951 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3455fc42-adf5-4265-9c27-30e990a9f2f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eaf19a259c2741cea375a38763e33cdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:26:51.238626 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3455fc42-adf5-4265-9c27-30e990a9f2f1 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:26:51.238909 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3455fc42-adf5-4265-9c27-30e990a9f2f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:26:51.239187 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3455fc42-adf5-4265-9c27-30e990a9f2f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:26:51.240815 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-3455fc42-adf5-4265-9c27-30e990a9f2f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:26:51.318405 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:26:51.366702 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3455fc42-adf5-4265-9c27-30e990a9f2f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:26:51.368186 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3455fc42-adf5-4265-9c27-30e990a9f2f1 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:26:51.391490 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 660/1331] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:26:51 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:26:54.271271 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993d2de3-de4d-4b87-bcde-e74dd12bb9fb None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:26:54.271825 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993d2de3-de4d-4b87-bcde-e74dd12bb9fb None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:26:54.271825 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993d2de3-de4d-4b87-bcde-e74dd12bb9fb None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:26:54.273219 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 672/1332] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:26:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:26:54.278810 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a157e2-03b2-4188-bde2-f06c05f986c8 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:26:54.279227 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a157e2-03b2-4188-bde2-f06c05f986c8 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:26:54.279360 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a157e2-03b2-4188-bde2-f06c05f986c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:26:54.343788 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-22a157e2-03b2-4188-bde2-f06c05f986c8 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:26:54.344997 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-22a157e2-03b2-4188-bde2-f06c05f986c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:26:54.440770 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 661/1333] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:26:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:26:54.481586 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993d2de3-de4d-4b87-bcde-e74dd12bb9fb None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:26:54.489213 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-993d2de3-de4d-4b87-bcde-e74dd12bb9fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:26:54.571247 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:26:54.605509 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993d2de3-de4d-4b87-bcde-e74dd12bb9fb None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:26:54.679404 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d9a719-c36b-4a5b-886d-dd233511c4e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eaf19a259c2741cea375a38763e33cdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:26:54.680049 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d9a719-c36b-4a5b-886d-dd233511c4e8 service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:26:54.680381 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d9a719-c36b-4a5b-886d-dd233511c4e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:26:54.680660 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d9a719-c36b-4a5b-886d-dd233511c4e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:26:54.682334 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-f5d9a719-c36b-4a5b-886d-dd233511c4e8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:26:54.791717 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d9a719-c36b-4a5b-886d-dd233511c4e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:26:54.793163 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d9a719-c36b-4a5b-886d-dd233511c4e8 service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:26:54.803654 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:26:54.815938 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 673/1334] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:26:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:26:56.869818 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac084f2-cffb-499f-8bbd-b2106dd3a219 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:26:56.870590 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac084f2-cffb-499f-8bbd-b2106dd3a219 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:26:56.870590 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac084f2-cffb-499f-8bbd-b2106dd3a219 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:26:56.871895 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 662/1335] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:26:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:26:56.877174 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a8dbdc-af3d-4094-9b79-b0d5c4aefe5f None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:26:56.877585 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a8dbdc-af3d-4094-9b79-b0d5c4aefe5f None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:26:56.877724 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a8dbdc-af3d-4094-9b79-b0d5c4aefe5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:26:56.943103 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-29a8dbdc-af3d-4094-9b79-b0d5c4aefe5f None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:26:56.944304 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-29a8dbdc-af3d-4094-9b79-b0d5c4aefe5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:26:57.037120 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 674/1336] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:26:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 02:26:57.078049 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac084f2-cffb-499f-8bbd-b2106dd3a219 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:26:57.079583 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-8ac084f2-cffb-499f-8bbd-b2106dd3a219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:26:57.118043 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:26:57.179549 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac084f2-cffb-499f-8bbd-b2106dd3a219 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:26:57.250047 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d68ad6-d190-4b5a-8565-d6eb66049e24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eaf19a259c2741cea375a38763e33cdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:26:57.250705 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d68ad6-d190-4b5a-8565-d6eb66049e24 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:26:57.250984 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d68ad6-d190-4b5a-8565-d6eb66049e24 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:26:57.251256 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d68ad6-d190-4b5a-8565-d6eb66049e24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:26:57.252854 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-f3d68ad6-d190-4b5a-8565-d6eb66049e24 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:26:57.333954 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:26:57.359752 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d68ad6-d190-4b5a-8565-d6eb66049e24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:26:57.361348 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d68ad6-d190-4b5a-8565-d6eb66049e24 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:26:57.380785 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 663/1337] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:26:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:27:00.288575 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa63c1d-6a47-491a-b257-8266ff534db9 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:27:00.289152 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa63c1d-6a47-491a-b257-8266ff534db9 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:27:00.289152 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa63c1d-6a47-491a-b257-8266ff534db9 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:27:00.290582 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 675/1338] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:27:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:27:00.296482 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b6364d-4841-4dc9-bd38-15f5d08fe5af None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:27:00.296935 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b6364d-4841-4dc9-bd38-15f5d08fe5af None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:27:00.297059 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b6364d-4841-4dc9-bd38-15f5d08fe5af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:27:00.362265 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-b0b6364d-4841-4dc9-bd38-15f5d08fe5af None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:27:00.363463 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-b0b6364d-4841-4dc9-bd38-15f5d08fe5af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:27:00.459198 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 664/1339] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:27:00 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:27:00.496430 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa63c1d-6a47-491a-b257-8266ff534db9 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:27:00.497992 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-6aa63c1d-6a47-491a-b257-8266ff534db9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:27:00.574638 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:27:00.601667 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa63c1d-6a47-491a-b257-8266ff534db9 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:27:00.676254 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ece2d9c-9f50-4d2c-8926-6c9bc826eee5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eaf19a259c2741cea375a38763e33cdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:27:00.676900 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ece2d9c-9f50-4d2c-8926-6c9bc826eee5 service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:27:00.677185 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ece2d9c-9f50-4d2c-8926-6c9bc826eee5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:27:00.677463 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ece2d9c-9f50-4d2c-8926-6c9bc826eee5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:27:00.679140 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-4ece2d9c-9f50-4d2c-8926-6c9bc826eee5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:27:00.787009 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ece2d9c-9f50-4d2c-8926-6c9bc826eee5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:27:00.788485 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ece2d9c-9f50-4d2c-8926-6c9bc826eee5 service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:27:00.807621 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 676/1340] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:27:00 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 02:27:05.185519 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e30569f-4b6b-4c0b-a7a5-cb6481bbcc42 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:27:05.186031 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e30569f-4b6b-4c0b-a7a5-cb6481bbcc42 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:27:05.186154 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e30569f-4b6b-4c0b-a7a5-cb6481bbcc42 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:27:05.187507 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 665/1341] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:27:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:27:05.192773 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157f3280-95fa-4af8-96e0-fc4504b2a511 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:27:05.199209 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157f3280-95fa-4af8-96e0-fc4504b2a511 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:27:05.199209 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157f3280-95fa-4af8-96e0-fc4504b2a511 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:27:05.199209 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:27:05.269227 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-157f3280-95fa-4af8-96e0-fc4504b2a511 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:27:05.270485 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-157f3280-95fa-4af8-96e0-fc4504b2a511 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:27:05.379429 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 677/1342] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:27:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:27:05.419583 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e30569f-4b6b-4c0b-a7a5-cb6481bbcc42 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:27:05.421148 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-5e30569f-4b6b-4c0b-a7a5-cb6481bbcc42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:27:05.470761 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:27:05.530390 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e30569f-4b6b-4c0b-a7a5-cb6481bbcc42 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:27:05.603355 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287ab89b-9817-4474-b2c0-25a55d393e06 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eaf19a259c2741cea375a38763e33cdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:27:05.604703 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287ab89b-9817-4474-b2c0-25a55d393e06 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:27:05.604984 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287ab89b-9817-4474-b2c0-25a55d393e06 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:27:05.605254 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287ab89b-9817-4474-b2c0-25a55d393e06 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:27:05.606926 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-287ab89b-9817-4474-b2c0-25a55d393e06 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:27:05.709137 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:27:05.725881 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287ab89b-9817-4474-b2c0-25a55d393e06 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:27:05.727370 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287ab89b-9817-4474-b2c0-25a55d393e06 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:27:05.749555 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 666/1343] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:27:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:27:08.728226 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa7e2e1-74fd-4287-935a-187d4b0b7918 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:27:08.728953 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa7e2e1-74fd-4287-935a-187d4b0b7918 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:27:08.728953 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa7e2e1-74fd-4287-935a-187d4b0b7918 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:27:08.730271 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 678/1344] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:27:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:27:08.735705 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58821009-2915-4ab6-88d1-6d020e76ff89 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:27:08.736151 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58821009-2915-4ab6-88d1-6d020e76ff89 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:27:08.736389 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58821009-2915-4ab6-88d1-6d020e76ff89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:27:08.800624 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-58821009-2915-4ab6-88d1-6d020e76ff89 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:27:08.801818 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-58821009-2915-4ab6-88d1-6d020e76ff89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:27:08.900472 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 667/1345] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:27:08 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:27:08.938574 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa7e2e1-74fd-4287-935a-187d4b0b7918 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:27:08.940164 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-dfa7e2e1-74fd-4287-935a-187d4b0b7918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:27:09.026834 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:27:09.044491 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa7e2e1-74fd-4287-935a-187d4b0b7918 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:27:09.118955 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6e2f1d-71b9-475c-8410-e979ca28d417 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eaf19a259c2741cea375a38763e33cdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:27:09.119829 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6e2f1d-71b9-475c-8410-e979ca28d417 service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:27:09.120113 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6e2f1d-71b9-475c-8410-e979ca28d417 service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:27:09.120385 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6e2f1d-71b9-475c-8410-e979ca28d417 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:27:09.122146 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-9c6e2f1d-71b9-475c-8410-e979ca28d417 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:27:09.231103 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6e2f1d-71b9-475c-8410-e979ca28d417 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:27:09.232561 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6e2f1d-71b9-475c-8410-e979ca28d417 service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:27:09.253052 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 679/1346] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:27:08 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:27:13.618359 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5501ba7-84f6-4c78-9079-4181faf844be None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:27:13.618900 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5501ba7-84f6-4c78-9079-4181faf844be None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:27:13.620442 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5501ba7-84f6-4c78-9079-4181faf844be None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:27:13.620804 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 668/1347] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:27:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:27:13.626953 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c48e1c8-dff4-4ee5-81ca-5733a6f1a288 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:27:13.627440 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c48e1c8-dff4-4ee5-81ca-5733a6f1a288 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:27:13.627707 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c48e1c8-dff4-4ee5-81ca-5733a6f1a288 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:27:13.642921 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:27:13.700968 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-9c48e1c8-dff4-4ee5-81ca-5733a6f1a288 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:27:13.702612 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-9c48e1c8-dff4-4ee5-81ca-5733a6f1a288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:27:13.811039 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 680/1348] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:27:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:27:13.850425 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5501ba7-84f6-4c78-9079-4181faf844be None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:27:13.852141 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-b5501ba7-84f6-4c78-9079-4181faf844be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:27:13.927964 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:27:13.976789 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5501ba7-84f6-4c78-9079-4181faf844be None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:27:14.054934 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c462474a-915d-4872-9b39-4f9ed817012f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eaf19a259c2741cea375a38763e33cdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:27:14.055587 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c462474a-915d-4872-9b39-4f9ed817012f service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:27:14.055871 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c462474a-915d-4872-9b39-4f9ed817012f service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:27:14.056291 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c462474a-915d-4872-9b39-4f9ed817012f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:27:14.057974 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-c462474a-915d-4872-9b39-4f9ed817012f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:27:14.167683 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:27:14.171525 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c462474a-915d-4872-9b39-4f9ed817012f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:27:14.172973 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c462474a-915d-4872-9b39-4f9ed817012f service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:27:14.192029 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 669/1349] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:27:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:27:19.196908 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4149c85-1147-4343-b8a4-8b72f368871f None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:27:19.197415 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4149c85-1147-4343-b8a4-8b72f368871f None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:27:19.197660 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4149c85-1147-4343-b8a4-8b72f368871f None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:27:19.199665 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 681/1350] 10.222.0.66 () {54 vars in 921 bytes} [Mon Jul 18 02:27:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:27:19.205061 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46084de7-f4b1-49b8-9f98-d3b7074b2401 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:27:19.205502 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46084de7-f4b1-49b8-9f98-d3b7074b2401 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:27:19.205598 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46084de7-f4b1-49b8-9f98-d3b7074b2401 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:27:19.349567 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-46084de7-f4b1-49b8-9f98-d3b7074b2401 None None] MFA Rules not processed for user `88936b31709b429f8b6777c673ed7b8a`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:27:19.350730 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-46084de7-f4b1-49b8-9f98-d3b7074b2401 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:27:19.489432 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 670/1351] 10.222.0.66 () {58 vars in 1018 bytes} [Mon Jul 18 02:27:19 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:27:23.556551 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d2765c-998c-4ed2-b505-007dfbd77f02 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:27:23.557080 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d2765c-998c-4ed2-b505-007dfbd77f02 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:27:23.557164 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d2765c-998c-4ed2-b505-007dfbd77f02 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:27:23.558654 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 682/1352] 10.222.0.66 () {54 vars in 921 bytes} [Mon Jul 18 02:27:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:27:23.564030 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c37b0e-2320-4da4-85d6-31729572ea45 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:27:23.564463 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c37b0e-2320-4da4-85d6-31729572ea45 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:27:23.570805 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c37b0e-2320-4da4-85d6-31729572ea45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:27:23.630322 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:27:23.654232 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-b3c37b0e-2320-4da4-85d6-31729572ea45 None None] MFA Rules not processed for user `88936b31709b429f8b6777c673ed7b8a`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:27:23.655442 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-b3c37b0e-2320-4da4-85d6-31729572ea45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:27:23.769046 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 671/1353] 10.222.0.66 () {58 vars in 1018 bytes} [Mon Jul 18 02:27:23 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:27:23.808952 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=59 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:27:25.116880 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748f135e-1e8c-4d83-bfac-cc0295e12740 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:27:25.117519 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748f135e-1e8c-4d83-bfac-cc0295e12740 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:27:25.117519 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748f135e-1e8c-4d83-bfac-cc0295e12740 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:27:25.119077 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 683/1354] 10.222.0.66 () {54 vars in 921 bytes} [Mon Jul 18 02:27:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:27:25.124592 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281e3ae4-9c98-4559-8eb5-bf4ef0cd8381 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:27:25.125056 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281e3ae4-9c98-4559-8eb5-bf4ef0cd8381 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:27:25.125156 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281e3ae4-9c98-4559-8eb5-bf4ef0cd8381 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:27:25.210932 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-281e3ae4-9c98-4559-8eb5-bf4ef0cd8381 None None] MFA Rules not processed for user `88936b31709b429f8b6777c673ed7b8a`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:27:25.212666 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-281e3ae4-9c98-4559-8eb5-bf4ef0cd8381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:27:25.256725 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:27:25.332352 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 672/1355] 10.222.0.66 () {58 vars in 1018 bytes} [Mon Jul 18 02:27:25 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:27:28.746325 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c232ec-ac6f-4e73-99bf-0ddb96cb712f None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:27:28.747223 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c232ec-ac6f-4e73-99bf-0ddb96cb712f None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:27:28.747676 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c232ec-ac6f-4e73-99bf-0ddb96cb712f None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:27:28.749612 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 684/1356] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:27:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:27:28.755525 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f7a05c-ec73-4b1d-b54b-638f786a494a None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:27:28.756336 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f7a05c-ec73-4b1d-b54b-638f786a494a None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:27:28.756783 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f7a05c-ec73-4b1d-b54b-638f786a494a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:27:28.858351 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-84f7a05c-ec73-4b1d-b54b-638f786a494a None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:27:28.859806 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-84f7a05c-ec73-4b1d-b54b-638f786a494a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:27:28.973928 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 673/1357] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:27:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:27:29.030640 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82133007-0ea9-4002-bd3c-88619307ed82 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:27:29.031037 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82133007-0ea9-4002-bd3c-88619307ed82 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:27:29.031412 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82133007-0ea9-4002-bd3c-88619307ed82 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:27:29.032898 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 685/1358] 10.222.0.66 () {54 vars in 962 bytes} [Mon Jul 18 02:27:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:27:29.040458 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acb64b3-b0d5-47ed-8727-39f2a2a07ed5 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:27:29.040816 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acb64b3-b0d5-47ed-8727-39f2a2a07ed5 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:27:29.041260 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acb64b3-b0d5-47ed-8727-39f2a2a07ed5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:27:29.060145 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:27:29.125873 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-8acb64b3-b0d5-47ed-8727-39f2a2a07ed5 None None] MFA Rules not processed for user `80b138831bc54c7b986d50092d62d4af`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:27:29.127878 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-8acb64b3-b0d5-47ed-8727-39f2a2a07ed5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:27:29.254975 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 674/1359] 10.222.0.66 () {58 vars in 1059 bytes} [Mon Jul 18 02:27:29 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:27:29.263368 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82133007-0ea9-4002-bd3c-88619307ed82 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:27:29.265015 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-82133007-0ea9-4002-bd3c-88619307ed82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:27:29.418288 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82133007-0ea9-4002-bd3c-88619307ed82 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:27:29.499336 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:27:29.544988 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9d8283-5fbf-43c7-b7f2-b6ecd3cd9714 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80b138831bc54c7b986d50092d62d4af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:27:29.545772 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9d8283-5fbf-43c7-b7f2-b6ecd3cd9714 service nova] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:27:29.546190 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9d8283-5fbf-43c7-b7f2-b6ecd3cd9714 service nova] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:27:29.546564 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9d8283-5fbf-43c7-b7f2-b6ecd3cd9714 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:27:29.548306 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-2e9d8283-5fbf-43c7-b7f2-b6ecd3cd9714 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:27:29.647675 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9d8283-5fbf-43c7-b7f2-b6ecd3cd9714 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:27:29.649795 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9d8283-5fbf-43c7-b7f2-b6ecd3cd9714 service nova] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:27:29.669994 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 686/1360] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:27:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:27:31.918035 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c855d5-1ebc-4745-aad8-a575f3fc8dae None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:27:31.918035 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c855d5-1ebc-4745-aad8-a575f3fc8dae None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:27:31.919592 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c855d5-1ebc-4745-aad8-a575f3fc8dae None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:27:31.922636 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 675/1361] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:27:31 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:27:31.925073 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1db17e-001c-49af-97df-f609315e5d80 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:27:31.925555 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1db17e-001c-49af-97df-f609315e5d80 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:27:31.925872 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1db17e-001c-49af-97df-f609315e5d80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:27:31.996729 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-af1db17e-001c-49af-97df-f609315e5d80 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:27:31.997922 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-af1db17e-001c-49af-97df-f609315e5d80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:27:32.037535 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:27:32.100118 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 687/1362] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:27:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:27:32.146757 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c855d5-1ebc-4745-aad8-a575f3fc8dae None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:27:32.148140 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-41c855d5-1ebc-4745-aad8-a575f3fc8dae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:27:32.222830 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:27:32.318612 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c855d5-1ebc-4745-aad8-a575f3fc8dae None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:27:32.488613 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3585fe5a-b317-42fd-a794-02737c294992 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80b138831bc54c7b986d50092d62d4af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:27:32.489463 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3585fe5a-b317-42fd-a794-02737c294992 service nova] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:27:32.489816 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3585fe5a-b317-42fd-a794-02737c294992 service nova] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:27:32.490179 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3585fe5a-b317-42fd-a794-02737c294992 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:27:32.493913 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-3585fe5a-b317-42fd-a794-02737c294992 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:27:32.520696 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:27:32.597066 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3585fe5a-b317-42fd-a794-02737c294992 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:27:32.598893 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3585fe5a-b317-42fd-a794-02737c294992 service nova] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:27:32.623022 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 676/1363] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:27:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 477 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:27:35.123514 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dafdf36-def3-4835-a9d4-559c433c20bb None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:27:35.124201 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dafdf36-def3-4835-a9d4-559c433c20bb None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:27:35.124201 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dafdf36-def3-4835-a9d4-559c433c20bb None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:27:35.125554 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 688/1364] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:27:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:27:35.131187 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c863810-4fac-4b68-acfc-d20b0f9c6ead None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:27:35.131660 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c863810-4fac-4b68-acfc-d20b0f9c6ead None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:27:35.131746 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c863810-4fac-4b68-acfc-d20b0f9c6ead None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:27:35.202914 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-5c863810-4fac-4b68-acfc-d20b0f9c6ead None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:27:35.207048 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-5c863810-4fac-4b68-acfc-d20b0f9c6ead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:27:35.308026 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:27:35.310997 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 677/1365] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:27:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:27:35.366118 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e62efe-bae9-4423-909e-4f6b43aa8f08 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:27:35.366654 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e62efe-bae9-4423-909e-4f6b43aa8f08 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:27:35.367111 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e62efe-bae9-4423-909e-4f6b43aa8f08 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:27:35.369052 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 689/1366] 10.222.0.66 () {54 vars in 962 bytes} [Mon Jul 18 02:27:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:27:35.374701 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63bbc76-5b3a-414f-8c42-56e609c2ba1d None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:27:35.374998 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63bbc76-5b3a-414f-8c42-56e609c2ba1d None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:27:35.375362 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63bbc76-5b3a-414f-8c42-56e609c2ba1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:27:35.468826 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-d63bbc76-5b3a-414f-8c42-56e609c2ba1d None None] MFA Rules not processed for user `80b138831bc54c7b986d50092d62d4af`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:27:35.470064 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-d63bbc76-5b3a-414f-8c42-56e609c2ba1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:27:35.608922 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 678/1367] 10.222.0.66 () {58 vars in 1059 bytes} [Mon Jul 18 02:27:35 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:27:35.619253 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e62efe-bae9-4423-909e-4f6b43aa8f08 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:27:35.620792 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-37e62efe-bae9-4423-909e-4f6b43aa8f08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:27:35.777898 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e62efe-bae9-4423-909e-4f6b43aa8f08 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:27:35.799085 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:27:35.896727 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab34e53-7603-4cab-ad4a-761195d905c4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80b138831bc54c7b986d50092d62d4af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:27:35.897385 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab34e53-7603-4cab-ad4a-761195d905c4 service nova] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:27:35.897670 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab34e53-7603-4cab-ad4a-761195d905c4 service nova] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:27:35.897950 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab34e53-7603-4cab-ad4a-761195d905c4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:27:35.900363 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-cab34e53-7603-4cab-ad4a-761195d905c4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:27:35.997481 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab34e53-7603-4cab-ad4a-761195d905c4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:27:35.999131 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab34e53-7603-4cab-ad4a-761195d905c4 service nova] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:27:36.019553 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 690/1368] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:27:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:27:36.949941 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860b2b65-3ef1-437f-98b4-5ba694035090 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:27:36.950527 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860b2b65-3ef1-437f-98b4-5ba694035090 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:27:36.950527 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860b2b65-3ef1-437f-98b4-5ba694035090 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:27:36.951963 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 679/1369] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:27:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:27:36.957319 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578cd8b6-44f3-40f9-8449-ebc7f3f4f79a None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:27:36.957812 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578cd8b6-44f3-40f9-8449-ebc7f3f4f79a None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:27:36.957906 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578cd8b6-44f3-40f9-8449-ebc7f3f4f79a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:27:36.991426 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:27:37.027541 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-578cd8b6-44f3-40f9-8449-ebc7f3f4f79a None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:27:37.030374 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-578cd8b6-44f3-40f9-8449-ebc7f3f4f79a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:27:37.142130 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 691/1370] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:27:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:27:37.491873 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5703c909-f4fc-4c24-94fe-17d9d688826b None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:27:37.491873 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5703c909-f4fc-4c24-94fe-17d9d688826b None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:27:37.491873 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5703c909-f4fc-4c24-94fe-17d9d688826b None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:27:37.494337 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 680/1371] 10.222.0.66 () {54 vars in 964 bytes} [Mon Jul 18 02:27:37 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:27:37.499474 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc76c869-230e-4a95-a52f-84a5e43a6735 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:27:37.499834 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc76c869-230e-4a95-a52f-84a5e43a6735 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:27:37.500178 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc76c869-230e-4a95-a52f-84a5e43a6735 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:27:37.593163 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-bc76c869-230e-4a95-a52f-84a5e43a6735 None None] MFA Rules not processed for user `cd49388facdf446885e55b45ef91f01e`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:27:37.594903 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-bc76c869-230e-4a95-a52f-84a5e43a6735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:27:37.657602 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:27:37.689786 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 692/1372] 10.222.0.66 () {58 vars in 1061 bytes} [Mon Jul 18 02:27:37 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 02:27:37.698227 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5703c909-f4fc-4c24-94fe-17d9d688826b None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:27:37.699834 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-5703c909-f4fc-4c24-94fe-17d9d688826b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:27:37.753759 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:27:37.813199 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5703c909-f4fc-4c24-94fe-17d9d688826b None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:27:37.892517 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e421fe7-1493-466a-bad6-7f82e6251691 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd49388facdf446885e55b45ef91f01e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:27:37.893326 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e421fe7-1493-466a-bad6-7f82e6251691 service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:27:37.893711 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e421fe7-1493-466a-bad6-7f82e6251691 service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:27:37.894110 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e421fe7-1493-466a-bad6-7f82e6251691 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:27:37.895875 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-7e421fe7-1493-466a-bad6-7f82e6251691 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:27:37.999429 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:27:38.013171 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e421fe7-1493-466a-bad6-7f82e6251691 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:27:38.014666 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e421fe7-1493-466a-bad6-7f82e6251691 service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:27:38.034185 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 681/1373] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:27:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:27:38.264784 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099d61d3-0e82-45ab-8d93-28fd5c5a5221 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:27:38.265567 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099d61d3-0e82-45ab-8d93-28fd5c5a5221 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:27:38.265567 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099d61d3-0e82-45ab-8d93-28fd5c5a5221 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:27:38.270812 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 693/1374] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:27:38 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:27:38.271998 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3ca2d7-825f-48e9-a771-e6c60dc428fd None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:27:38.272172 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3ca2d7-825f-48e9-a771-e6c60dc428fd None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:27:38.272572 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3ca2d7-825f-48e9-a771-e6c60dc428fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:27:38.340358 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-5e3ca2d7-825f-48e9-a771-e6c60dc428fd None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:27:38.341708 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-5e3ca2d7-825f-48e9-a771-e6c60dc428fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:27:38.454017 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 682/1375] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:27:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 02:27:38.500097 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099d61d3-0e82-45ab-8d93-28fd5c5a5221 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:27:38.501811 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-099d61d3-0e82-45ab-8d93-28fd5c5a5221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:27:38.659124 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099d61d3-0e82-45ab-8d93-28fd5c5a5221 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:27:38.713779 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:27:38.774713 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21545c0f-643e-4583-8e8e-367341c9f299 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80b138831bc54c7b986d50092d62d4af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:27:38.775514 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21545c0f-643e-4583-8e8e-367341c9f299 service nova] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:27:38.775800 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21545c0f-643e-4583-8e8e-367341c9f299 service nova] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:27:38.776080 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21545c0f-643e-4583-8e8e-367341c9f299 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:27:38.777803 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-21545c0f-643e-4583-8e8e-367341c9f299 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:27:38.870478 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21545c0f-643e-4583-8e8e-367341c9f299 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:27:38.872076 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21545c0f-643e-4583-8e8e-367341c9f299 service nova] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:27:38.891187 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 694/1376] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:27:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:27:41.004674 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270e2f04-363e-48a7-8b76-c03e96078978 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:27:41.005207 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270e2f04-363e-48a7-8b76-c03e96078978 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:27:41.005300 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270e2f04-363e-48a7-8b76-c03e96078978 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:27:41.007343 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 683/1377] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:27:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:27:41.012734 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f1c05c-eb6c-4cc4-a202-553ed03abb11 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:27:41.013138 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f1c05c-eb6c-4cc4-a202-553ed03abb11 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:27:41.013297 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f1c05c-eb6c-4cc4-a202-553ed03abb11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:27:41.078898 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-19f1c05c-eb6c-4cc4-a202-553ed03abb11 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:27:41.080106 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-19f1c05c-eb6c-4cc4-a202-553ed03abb11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:27:41.087259 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:27:41.189999 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 695/1378] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:27:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:27:41.240399 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270e2f04-363e-48a7-8b76-c03e96078978 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:27:41.242216 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-270e2f04-363e-48a7-8b76-c03e96078978 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:27:41.310202 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:27:41.395541 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270e2f04-363e-48a7-8b76-c03e96078978 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:27:41.518233 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcff6f5-cb39-4535-aa9a-20e53a7918fb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80b138831bc54c7b986d50092d62d4af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:27:41.518889 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcff6f5-cb39-4535-aa9a-20e53a7918fb service nova] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:27:41.519168 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcff6f5-cb39-4535-aa9a-20e53a7918fb service nova] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:27:41.519471 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcff6f5-cb39-4535-aa9a-20e53a7918fb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:27:41.521113 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-9bcff6f5-cb39-4535-aa9a-20e53a7918fb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:27:41.544298 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:27:41.651599 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcff6f5-cb39-4535-aa9a-20e53a7918fb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:27:41.653464 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcff6f5-cb39-4535-aa9a-20e53a7918fb service nova] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:27:41.689742 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 684/1379] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:27:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:27:43.897767 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf9c9aa-df8e-478d-b5d8-ac8141a7be3b None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:27:43.898372 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf9c9aa-df8e-478d-b5d8-ac8141a7be3b None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:27:43.898600 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf9c9aa-df8e-478d-b5d8-ac8141a7be3b None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:27:43.900225 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 696/1380] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:27:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:27:43.905569 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d99400a-7144-4e24-a679-607d8807f798 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:27:43.906019 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d99400a-7144-4e24-a679-607d8807f798 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:27:43.906658 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d99400a-7144-4e24-a679-607d8807f798 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:27:43.976192 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-6d99400a-7144-4e24-a679-607d8807f798 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:27:43.977399 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-6d99400a-7144-4e24-a679-607d8807f798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:27:44.075248 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:27:44.081756 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 685/1381] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:27:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:27:44.127010 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf9c9aa-df8e-478d-b5d8-ac8141a7be3b None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:27:44.128596 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-fcf9c9aa-df8e-478d-b5d8-ac8141a7be3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:27:44.271739 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:27:44.282428 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf9c9aa-df8e-478d-b5d8-ac8141a7be3b None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:27:44.403436 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a35f89-01d6-48c6-b810-7626e16ae937 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80b138831bc54c7b986d50092d62d4af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:27:44.404092 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a35f89-01d6-48c6-b810-7626e16ae937 service nova] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:27:44.404373 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a35f89-01d6-48c6-b810-7626e16ae937 service nova] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:27:44.404645 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a35f89-01d6-48c6-b810-7626e16ae937 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:27:44.406328 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-42a35f89-01d6-48c6-b810-7626e16ae937 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:27:44.501102 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a35f89-01d6-48c6-b810-7626e16ae937 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:27:44.502750 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a35f89-01d6-48c6-b810-7626e16ae937 service nova] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:27:44.514824 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:27:44.524167 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 697/1382] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:27:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:27:46.958798 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b0296e-812d-4926-921d-7a3d07880cb3 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:27:46.959344 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b0296e-812d-4926-921d-7a3d07880cb3 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:27:46.959344 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b0296e-812d-4926-921d-7a3d07880cb3 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:27:46.961734 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 686/1383] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:27:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:27:46.968029 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cece7601-0b93-4893-9f42-482804a428a8 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:27:46.968471 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cece7601-0b93-4893-9f42-482804a428a8 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:27:46.968604 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cece7601-0b93-4893-9f42-482804a428a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:27:47.039734 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-cece7601-0b93-4893-9f42-482804a428a8 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:27:47.041009 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-cece7601-0b93-4893-9f42-482804a428a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:27:47.139517 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 698/1384] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:27:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:27:47.183250 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b0296e-812d-4926-921d-7a3d07880cb3 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:27:47.184797 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-50b0296e-812d-4926-921d-7a3d07880cb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:27:47.348570 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b0296e-812d-4926-921d-7a3d07880cb3 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:27:47.429185 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:27:47.477861 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1143b5a-37d3-4ed2-b539-7539bd35a210 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80b138831bc54c7b986d50092d62d4af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:27:47.478549 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1143b5a-37d3-4ed2-b539-7539bd35a210 service nova] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:27:47.479047 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1143b5a-37d3-4ed2-b539-7539bd35a210 service nova] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:27:47.479345 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1143b5a-37d3-4ed2-b539-7539bd35a210 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:27:47.481020 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-e1143b5a-37d3-4ed2-b539-7539bd35a210 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:27:47.578255 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1143b5a-37d3-4ed2-b539-7539bd35a210 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:27:47.580483 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1143b5a-37d3-4ed2-b539-7539bd35a210 service nova] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:27:47.606389 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 687/1385] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:27:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:27:49.739465 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7416a58b-f26c-47a4-bb1a-7fdaec92a0fd None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:27:49.740130 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7416a58b-f26c-47a4-bb1a-7fdaec92a0fd None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:27:49.740130 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7416a58b-f26c-47a4-bb1a-7fdaec92a0fd None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:27:49.741997 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 699/1386] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:27:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:27:49.748735 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c640b24-0764-4ec8-adbf-4d9d8fa3bdcc None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:27:49.749200 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c640b24-0764-4ec8-adbf-4d9d8fa3bdcc None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:27:49.749420 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c640b24-0764-4ec8-adbf-4d9d8fa3bdcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:27:49.821894 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-6c640b24-0764-4ec8-adbf-4d9d8fa3bdcc None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:27:49.823153 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-6c640b24-0764-4ec8-adbf-4d9d8fa3bdcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:27:49.857107 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:27:49.929125 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 688/1387] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:27:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:27:49.973619 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7416a58b-f26c-47a4-bb1a-7fdaec92a0fd None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:27:49.976093 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-7416a58b-f26c-47a4-bb1a-7fdaec92a0fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:27:50.064392 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:27:50.148041 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7416a58b-f26c-47a4-bb1a-7fdaec92a0fd None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:27:50.284597 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a01a84-e194-4d08-b8fc-5e4196ec0411 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80b138831bc54c7b986d50092d62d4af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:27:50.285380 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a01a84-e194-4d08-b8fc-5e4196ec0411 service nova] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:27:50.285662 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a01a84-e194-4d08-b8fc-5e4196ec0411 service nova] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:27:50.286041 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a01a84-e194-4d08-b8fc-5e4196ec0411 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:27:50.288693 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-a7a01a84-e194-4d08-b8fc-5e4196ec0411 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:27:50.333920 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:27:50.394570 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a01a84-e194-4d08-b8fc-5e4196ec0411 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:27:50.396534 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a01a84-e194-4d08-b8fc-5e4196ec0411 service nova] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:27:50.418964 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 700/1388] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:27:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 02:27:52.682524 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bb45d2-35f5-4360-a81b-f041967e1e21 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:27:52.683087 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bb45d2-35f5-4360-a81b-f041967e1e21 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:27:52.683087 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bb45d2-35f5-4360-a81b-f041967e1e21 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:27:52.684620 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 689/1389] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:27:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:27:52.689651 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecc4d8a-ebfd-4f1c-82e2-dfa4091e15e7 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:27:52.690052 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecc4d8a-ebfd-4f1c-82e2-dfa4091e15e7 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:27:52.690245 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecc4d8a-ebfd-4f1c-82e2-dfa4091e15e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:27:52.754286 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-9ecc4d8a-ebfd-4f1c-82e2-dfa4091e15e7 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:27:52.755534 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-9ecc4d8a-ebfd-4f1c-82e2-dfa4091e15e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:27:52.850978 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 701/1390] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:27:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:27:52.895695 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bb45d2-35f5-4360-a81b-f041967e1e21 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:27:52.897404 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-24bb45d2-35f5-4360-a81b-f041967e1e21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:27:53.045843 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bb45d2-35f5-4360-a81b-f041967e1e21 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:27:53.063099 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:27:53.159014 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2cdb38-2324-417e-99a5-f92b42d96ee7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80b138831bc54c7b986d50092d62d4af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:27:53.159668 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2cdb38-2324-417e-99a5-f92b42d96ee7 service nova] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:27:53.159949 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2cdb38-2324-417e-99a5-f92b42d96ee7 service nova] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:27:53.160224 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2cdb38-2324-417e-99a5-f92b42d96ee7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:27:53.161870 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-2c2cdb38-2324-417e-99a5-f92b42d96ee7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:27:53.253031 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2cdb38-2324-417e-99a5-f92b42d96ee7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:27:53.254675 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2cdb38-2324-417e-99a5-f92b42d96ee7 service nova] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:27:53.273765 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 690/1391] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:27:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:27:53.461926 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef6c605-56e6-4e59-9a2d-df80f028bd6b None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:27:53.462252 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef6c605-56e6-4e59-9a2d-df80f028bd6b None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:27:53.462536 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef6c605-56e6-4e59-9a2d-df80f028bd6b None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:27:53.464076 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 702/1392] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:27:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:27:53.469982 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca0fa2f-9dba-4b3a-8166-e51294bf3d59 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:27:53.470302 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca0fa2f-9dba-4b3a-8166-e51294bf3d59 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:27:53.470580 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca0fa2f-9dba-4b3a-8166-e51294bf3d59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:27:53.498123 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:27:53.546552 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-7ca0fa2f-9dba-4b3a-8166-e51294bf3d59 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:27:53.547795 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-7ca0fa2f-9dba-4b3a-8166-e51294bf3d59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:27:53.653501 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 691/1393] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:27:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:27:53.710330 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bf49d8-9ae9-4100-a8da-a53d44a99480 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:27:53.710805 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bf49d8-9ae9-4100-a8da-a53d44a99480 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:27:53.710922 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bf49d8-9ae9-4100-a8da-a53d44a99480 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:27:53.712343 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 703/1394] 10.222.0.66 () {54 vars in 961 bytes} [Mon Jul 18 02:27:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:27:53.717528 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefc1022-4930-4af8-8208-36f934451553 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:27:53.717995 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefc1022-4930-4af8-8208-36f934451553 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:27:53.718171 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefc1022-4930-4af8-8208-36f934451553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:27:53.795954 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-cefc1022-4930-4af8-8208-36f934451553 None None] MFA Rules not processed for user `e1391b81e7234bc18e22b9e84f231256`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:27:53.797147 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-cefc1022-4930-4af8-8208-36f934451553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:27:53.839311 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:27:53.876155 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 692/1395] 10.222.0.66 () {58 vars in 1058 bytes} [Mon Jul 18 02:27:53 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:27:53.884496 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bf49d8-9ae9-4100-a8da-a53d44a99480 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:27:53.885988 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-e5bf49d8-9ae9-4100-a8da-a53d44a99480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:27:53.900535 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:27:53.993969 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bf49d8-9ae9-4100-a8da-a53d44a99480 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:27:54.071455 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98223fb5-fa2d-4289-93bd-232eac741442 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1391b81e7234bc18e22b9e84f231256', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:27:54.072110 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98223fb5-fa2d-4289-93bd-232eac741442 service glance] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:27:54.072396 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98223fb5-fa2d-4289-93bd-232eac741442 service glance] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:27:54.072701 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98223fb5-fa2d-4289-93bd-232eac741442 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:27:54.074456 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-98223fb5-fa2d-4289-93bd-232eac741442 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:27:54.145163 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:27:54.185571 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98223fb5-fa2d-4289-93bd-232eac741442 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:27:54.187120 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98223fb5-fa2d-4289-93bd-232eac741442 service glance] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:27:54.205962 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 704/1396] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:27:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:27:55.055766 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb75b9f7-0ed1-4386-9c83-7c10a6441ed1 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:27:55.056297 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb75b9f7-0ed1-4386-9c83-7c10a6441ed1 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:27:55.056609 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb75b9f7-0ed1-4386-9c83-7c10a6441ed1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:27:55.126394 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-fb75b9f7-0ed1-4386-9c83-7c10a6441ed1 None None] MFA Rules not processed for user `768f872096584d4093fc616d547aabb6`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:27:55.127612 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-fb75b9f7-0ed1-4386-9c83-7c10a6441ed1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:27:55.222662 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 693/1397] 10.222.0.66 () {58 vars in 1009 bytes} [Mon Jul 18 02:27:55 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:27:55.270545 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef9cc95-e4eb-4d96-82ae-37adb7759ee2 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:27:55.271027 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef9cc95-e4eb-4d96-82ae-37adb7759ee2 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:27:55.271347 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef9cc95-e4eb-4d96-82ae-37adb7759ee2 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:27:55.273695 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 705/1398] 10.222.0.66 () {54 vars in 943 bytes} [Mon Jul 18 02:27:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:27:55.279905 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677db682-009d-4651-9dda-fb85c870f846 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:27:55.280198 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677db682-009d-4651-9dda-fb85c870f846 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:27:55.280475 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677db682-009d-4651-9dda-fb85c870f846 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:27:55.348198 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:27:55.355850 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-677db682-009d-4651-9dda-fb85c870f846 None None] MFA Rules not processed for user `b4300622cbd347dd8f5227bb960e98d0`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:27:55.357122 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-677db682-009d-4651-9dda-fb85c870f846 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:27:55.446884 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af71f220-1313-4f9f-afb6-9248b3c63c7d None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:27:55.447171 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af71f220-1313-4f9f-afb6-9248b3c63c7d None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:27:55.447448 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af71f220-1313-4f9f-afb6-9248b3c63c7d None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:27:55.449019 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 706/1399] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:27:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 02:27:55.453651 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9908970-2f5e-4c74-8a89-b6d413d5b10a None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:27:55.453935 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9908970-2f5e-4c74-8a89-b6d413d5b10a None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:27:55.454248 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9908970-2f5e-4c74-8a89-b6d413d5b10a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:27:55.467867 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 694/1400] 10.222.0.66 () {58 vars in 1040 bytes} [Mon Jul 18 02:27:55 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 02:27:55.477436 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677db682-009d-4651-9dda-fb85c870f846 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:27:55.487039 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677db682-009d-4651-9dda-fb85c870f846 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:27:55.521563 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-b9908970-2f5e-4c74-8a89-b6d413d5b10a None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:27:55.522823 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-b9908970-2f5e-4c74-8a89-b6d413d5b10a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:27:55.597376 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac0c02a-70db-44ce-86e9-efa1c8fae559 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4300622cbd347dd8f5227bb960e98d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:27:55.598016 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac0c02a-70db-44ce-86e9-efa1c8fae559 service swift] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:27:55.598288 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac0c02a-70db-44ce-86e9-efa1c8fae559 service swift] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:27:55.598522 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac0c02a-70db-44ce-86e9-efa1c8fae559 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:27:55.600552 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-6ac0c02a-70db-44ce-86e9-efa1c8fae559 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:27:55.603351 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:27:55.626044 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 707/1401] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:27:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 02:27:55.673676 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9908970-2f5e-4c74-8a89-b6d413d5b10a None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:27:55.675040 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-b9908970-2f5e-4c74-8a89-b6d413d5b10a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:27:55.704698 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:27:55.794711 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9908970-2f5e-4c74-8a89-b6d413d5b10a None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:27:55.808002 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac0c02a-70db-44ce-86e9-efa1c8fae559 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:27:55.809597 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac0c02a-70db-44ce-86e9-efa1c8fae559 service swift] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:27:55.817569 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 695/1402] 10.222.0.66 () {60 vars in 1384 bytes} [Mon Jul 18 02:27:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 18 02:27:55.920861 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486a9e41-a382-4231-a34c-0b2303c6a658 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80b138831bc54c7b986d50092d62d4af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:27:55.922142 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486a9e41-a382-4231-a34c-0b2303c6a658 service nova] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:27:55.922596 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486a9e41-a382-4231-a34c-0b2303c6a658 service nova] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:27:55.923066 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486a9e41-a382-4231-a34c-0b2303c6a658 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:27:55.924893 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-486a9e41-a382-4231-a34c-0b2303c6a658 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:27:55.943714 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:27:56.021912 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486a9e41-a382-4231-a34c-0b2303c6a658 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:27:56.021912 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486a9e41-a382-4231-a34c-0b2303c6a658 service nova] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:27:56.035698 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 708/1403] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:27:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 02:27:58.404055 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecc926f-847c-4b93-b618-606de0925a5c None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:27:58.404685 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecc926f-847c-4b93-b618-606de0925a5c None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:27:58.404685 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecc926f-847c-4b93-b618-606de0925a5c None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:27:58.406131 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 696/1404] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:27:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:27:58.411490 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b6c605-b09b-4b24-838a-f0aaddfa75d5 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:27:58.412518 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b6c605-b09b-4b24-838a-f0aaddfa75d5 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:27:58.412812 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b6c605-b09b-4b24-838a-f0aaddfa75d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:27:58.485901 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-b2b6c605-b09b-4b24-838a-f0aaddfa75d5 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:27:58.488287 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-b2b6c605-b09b-4b24-838a-f0aaddfa75d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:27:58.591806 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:27:58.607184 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 709/1405] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:27:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:27:58.646679 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecc926f-847c-4b93-b618-606de0925a5c None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:27:58.648578 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-1ecc926f-847c-4b93-b618-606de0925a5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:27:58.826410 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:27:58.830521 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecc926f-847c-4b93-b618-606de0925a5c None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:27:58.990466 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77915caf-cca8-432e-8d2a-bbe85b0e24d3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80b138831bc54c7b986d50092d62d4af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:27:58.991154 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77915caf-cca8-432e-8d2a-bbe85b0e24d3 service nova] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:27:58.991417 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77915caf-cca8-432e-8d2a-bbe85b0e24d3 service nova] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:27:58.991697 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77915caf-cca8-432e-8d2a-bbe85b0e24d3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:27:58.993358 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-77915caf-cca8-432e-8d2a-bbe85b0e24d3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:27:59.121200 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77915caf-cca8-432e-8d2a-bbe85b0e24d3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:27:59.124055 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77915caf-cca8-432e-8d2a-bbe85b0e24d3 service nova] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:27:59.145934 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 697/1406] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:27:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 500 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:28:01.340599 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a9d3ed-2eb1-470c-ad3e-47c2ab564727 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:28:01.341232 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a9d3ed-2eb1-470c-ad3e-47c2ab564727 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:28:01.341232 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a9d3ed-2eb1-470c-ad3e-47c2ab564727 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:28:01.342929 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 710/1407] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:28:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:28:01.348403 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16624b45-85be-4179-a1a8-5077a9191544 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:28:01.348932 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16624b45-85be-4179-a1a8-5077a9191544 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:28:01.349039 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16624b45-85be-4179-a1a8-5077a9191544 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:28:01.353293 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:28:01.426640 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-16624b45-85be-4179-a1a8-5077a9191544 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:28:01.428553 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-16624b45-85be-4179-a1a8-5077a9191544 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:28:01.543020 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 698/1408] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:28:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:28:01.591153 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a9d3ed-2eb1-470c-ad3e-47c2ab564727 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:28:01.592714 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-07a9d3ed-2eb1-470c-ad3e-47c2ab564727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:28:01.753400 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a9d3ed-2eb1-470c-ad3e-47c2ab564727 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:28:01.825937 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:28:01.875013 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c485b17-1197-45c7-8932-e55827214b26 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80b138831bc54c7b986d50092d62d4af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:28:01.875941 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c485b17-1197-45c7-8932-e55827214b26 service nova] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:28:01.876226 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c485b17-1197-45c7-8932-e55827214b26 service nova] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:28:01.876497 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c485b17-1197-45c7-8932-e55827214b26 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:28:01.878168 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-0c485b17-1197-45c7-8932-e55827214b26 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:28:01.976339 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c485b17-1197-45c7-8932-e55827214b26 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:28:01.978235 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c485b17-1197-45c7-8932-e55827214b26 service nova] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:28:01.998597 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 711/1409] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:28:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:28:11.540569 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=64 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:28:11.995894 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=64 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:28:19.718304 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6956c329-4c96-42da-bce6-86e284e390c5 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:28:19.718891 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6956c329-4c96-42da-bce6-86e284e390c5 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:28:19.718891 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6956c329-4c96-42da-bce6-86e284e390c5 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:28:19.720229 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 699/1410] 10.222.0.66 () {54 vars in 918 bytes} [Mon Jul 18 02:28:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:28:19.725891 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b5117c-1670-447e-a0ec-5241cfecb30d None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:28:19.726429 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b5117c-1670-447e-a0ec-5241cfecb30d None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:28:19.726546 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b5117c-1670-447e-a0ec-5241cfecb30d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:28:19.801273 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-a7b5117c-1670-447e-a0ec-5241cfecb30d None None] MFA Rules not processed for user `88936b31709b429f8b6777c673ed7b8a`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:28:19.802440 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-a7b5117c-1670-447e-a0ec-5241cfecb30d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:28:19.904128 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 712/1411] 10.222.0.66 () {58 vars in 1015 bytes} [Mon Jul 18 02:28:19 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:28:19.919201 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7c0403-18b8-4cf1-a079-a5760fcb3859 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:28:19.919490 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7c0403-18b8-4cf1-a079-a5760fcb3859 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:28:19.919767 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7c0403-18b8-4cf1-a079-a5760fcb3859 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:28:19.921163 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 700/1412] 10.222.0.66 () {54 vars in 918 bytes} [Mon Jul 18 02:28:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:28:19.925652 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1150ad3d-044f-4fc7-844a-e7eb6cd43b66 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:28:19.926493 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1150ad3d-044f-4fc7-844a-e7eb6cd43b66 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:28:19.926775 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1150ad3d-044f-4fc7-844a-e7eb6cd43b66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:28:19.996519 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:28:19.999495 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-1150ad3d-044f-4fc7-844a-e7eb6cd43b66 None None] MFA Rules not processed for user `80b138831bc54c7b986d50092d62d4af`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:28:20.000693 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-1150ad3d-044f-4fc7-844a-e7eb6cd43b66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:28:20.114514 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 713/1413] 10.222.0.66 () {58 vars in 1015 bytes} [Mon Jul 18 02:28:19 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:28:22.608230 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248e14f7-a991-48f4-8a6d-7df86240343f None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:28:22.608752 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248e14f7-a991-48f4-8a6d-7df86240343f None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:28:22.608752 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248e14f7-a991-48f4-8a6d-7df86240343f None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:28:22.610133 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 701/1414] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 02:28:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:28:22.615576 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b76e31-35cf-4d68-b6ce-63857229a280 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:28:22.616046 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b76e31-35cf-4d68-b6ce-63857229a280 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:28:22.616160 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b76e31-35cf-4d68-b6ce-63857229a280 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:28:22.680952 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-f6b76e31-35cf-4d68-b6ce-63857229a280 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:28:22.682185 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-f6b76e31-35cf-4d68-b6ce-63857229a280 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:28:22.736816 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:28:22.777763 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 714/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 02:28:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:28:22.820537 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248e14f7-a991-48f4-8a6d-7df86240343f None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:28:22.822076 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-248e14f7-a991-48f4-8a6d-7df86240343f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:28:22.965492 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248e14f7-a991-48f4-8a6d-7df86240343f None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:28:23.035993 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:28:23.075051 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cfc205-de4d-4256-ade6-cc9ad2c169ab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80b138831bc54c7b986d50092d62d4af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:28:23.075697 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cfc205-de4d-4256-ade6-cc9ad2c169ab service nova] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:28:23.075978 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cfc205-de4d-4256-ade6-cc9ad2c169ab service nova] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:28:23.076251 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cfc205-de4d-4256-ade6-cc9ad2c169ab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:28:23.077870 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-67cfc205-de4d-4256-ade6-cc9ad2c169ab service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:28:23.165414 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cfc205-de4d-4256-ade6-cc9ad2c169ab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:28:23.167306 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cfc205-de4d-4256-ade6-cc9ad2c169ab service nova] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:28:23.185582 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 702/1416] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:28:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:28:24.078106 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877e10c3-7452-4e43-9057-ca1f7b1abe3b None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:28:24.078681 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877e10c3-7452-4e43-9057-ca1f7b1abe3b None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:28:24.078789 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877e10c3-7452-4e43-9057-ca1f7b1abe3b None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:28:24.080206 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 715/1417] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 02:28:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:28:24.085734 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7c9991-fb36-49fc-b467-5c7e849d3553 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:28:24.086186 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7c9991-fb36-49fc-b467-5c7e849d3553 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:28:24.086326 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7c9991-fb36-49fc-b467-5c7e849d3553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:28:24.160363 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-ca7c9991-fb36-49fc-b467-5c7e849d3553 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:28:24.161594 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-ca7c9991-fb36-49fc-b467-5c7e849d3553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:28:24.187793 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:28:24.255315 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 703/1418] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 02:28:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:28:24.405992 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877e10c3-7452-4e43-9057-ca1f7b1abe3b None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:28:24.407566 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-877e10c3-7452-4e43-9057-ca1f7b1abe3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:28:24.551573 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877e10c3-7452-4e43-9057-ca1f7b1abe3b None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:28:24.590242 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:28:24.664148 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a148e5-54cc-4591-9523-1838173d9b97 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80b138831bc54c7b986d50092d62d4af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:28:24.665071 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a148e5-54cc-4591-9523-1838173d9b97 service nova] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:28:24.665541 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a148e5-54cc-4591-9523-1838173d9b97 service nova] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:28:24.666139 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a148e5-54cc-4591-9523-1838173d9b97 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:28:24.668247 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-41a148e5-54cc-4591-9523-1838173d9b97 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:28:24.756343 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a148e5-54cc-4591-9523-1838173d9b97 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:28:24.757957 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a148e5-54cc-4591-9523-1838173d9b97 service nova] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:28:24.776393 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 716/1419] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:28:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:28:25.656947 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d24dbd0-257e-446d-94e8-c568d4e14b60 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:28:25.657511 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d24dbd0-257e-446d-94e8-c568d4e14b60 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:28:25.657511 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d24dbd0-257e-446d-94e8-c568d4e14b60 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:28:25.658930 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 704/1420] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 02:28:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:28:25.663925 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48936fe6-5889-42b9-98c4-ed1043ca6d87 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:28:25.664341 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48936fe6-5889-42b9-98c4-ed1043ca6d87 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:28:25.664462 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48936fe6-5889-42b9-98c4-ed1043ca6d87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:28:25.717503 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:28:25.739790 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-48936fe6-5889-42b9-98c4-ed1043ca6d87 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:28:25.741008 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-48936fe6-5889-42b9-98c4-ed1043ca6d87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:28:25.839006 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 717/1421] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 02:28:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:28:25.990846 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d24dbd0-257e-446d-94e8-c568d4e14b60 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:28:25.992342 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-2d24dbd0-257e-446d-94e8-c568d4e14b60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:28:26.135015 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d24dbd0-257e-446d-94e8-c568d4e14b60 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:28:26.147466 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:28:26.245833 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb2ba01-e7b2-4d27-b470-df0e6a807813 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80b138831bc54c7b986d50092d62d4af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:28:26.246498 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb2ba01-e7b2-4d27-b470-df0e6a807813 service nova] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:28:26.246780 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb2ba01-e7b2-4d27-b470-df0e6a807813 service nova] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:28:26.247054 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb2ba01-e7b2-4d27-b470-df0e6a807813 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:28:26.248674 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-5eb2ba01-e7b2-4d27-b470-df0e6a807813 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:28:26.336377 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb2ba01-e7b2-4d27-b470-df0e6a807813 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:28:26.337979 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb2ba01-e7b2-4d27-b470-df0e6a807813 service nova] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:28:26.355910 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 705/1422] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:28:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:28:27.215491 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2965b9e9-e39f-42e3-b684-2fdc4128ca31 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:28:27.216049 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2965b9e9-e39f-42e3-b684-2fdc4128ca31 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:28:27.216049 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2965b9e9-e39f-42e3-b684-2fdc4128ca31 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:28:27.217432 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 718/1423] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 02:28:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:28:27.223031 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebad557-d3b5-4b95-b440-3d8c61edd677 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:28:27.223509 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebad557-d3b5-4b95-b440-3d8c61edd677 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:28:27.223610 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebad557-d3b5-4b95-b440-3d8c61edd677 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:28:27.241911 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:28:27.299132 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-3ebad557-d3b5-4b95-b440-3d8c61edd677 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:28:27.300552 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-3ebad557-d3b5-4b95-b440-3d8c61edd677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:28:27.395542 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 706/1424] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 02:28:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:28:27.545658 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2965b9e9-e39f-42e3-b684-2fdc4128ca31 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:28:27.547131 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-2965b9e9-e39f-42e3-b684-2fdc4128ca31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:28:27.658034 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:28:27.694132 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2965b9e9-e39f-42e3-b684-2fdc4128ca31 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:28:27.805283 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f6e180-814f-48ea-950b-f89675b1901b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80b138831bc54c7b986d50092d62d4af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:28:27.805855 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f6e180-814f-48ea-950b-f89675b1901b service nova] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:28:27.806170 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f6e180-814f-48ea-950b-f89675b1901b service nova] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:28:27.806452 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f6e180-814f-48ea-950b-f89675b1901b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:28:27.808084 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-73f6e180-814f-48ea-950b-f89675b1901b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:28:27.884080 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:28:27.897287 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f6e180-814f-48ea-950b-f89675b1901b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:28:27.898991 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f6e180-814f-48ea-950b-f89675b1901b service nova] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:28:27.917554 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 719/1425] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:28:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:28:28.779784 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27140a4b-601c-4b7e-8a04-0cfa37366e3a None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:28:28.780339 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27140a4b-601c-4b7e-8a04-0cfa37366e3a None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:28:28.780339 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27140a4b-601c-4b7e-8a04-0cfa37366e3a None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:28:28.781708 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 707/1426] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 02:28:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:28:28.787361 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdcf49e-162f-42a5-aae9-cba1571ee11e None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:28:28.787844 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdcf49e-162f-42a5-aae9-cba1571ee11e None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:28:28.787951 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdcf49e-162f-42a5-aae9-cba1571ee11e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:28:28.865953 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-ebdcf49e-162f-42a5-aae9-cba1571ee11e None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:28:28.867112 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-ebdcf49e-162f-42a5-aae9-cba1571ee11e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:28:28.967359 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 720/1427] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 02:28:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:28:29.118065 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27140a4b-601c-4b7e-8a04-0cfa37366e3a None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:28:29.119515 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-27140a4b-601c-4b7e-8a04-0cfa37366e3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:28:29.202967 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:28:29.265179 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27140a4b-601c-4b7e-8a04-0cfa37366e3a None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:28:29.376279 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa00ddde-5125-490c-ae47-4fa7320f9329 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80b138831bc54c7b986d50092d62d4af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:28:29.376933 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa00ddde-5125-490c-ae47-4fa7320f9329 service nova] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:28:29.377217 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa00ddde-5125-490c-ae47-4fa7320f9329 service nova] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:28:29.377489 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa00ddde-5125-490c-ae47-4fa7320f9329 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:28:29.380052 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-fa00ddde-5125-490c-ae47-4fa7320f9329 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:28:29.430106 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:28:29.470643 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa00ddde-5125-490c-ae47-4fa7320f9329 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:28:29.472254 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa00ddde-5125-490c-ae47-4fa7320f9329 service nova] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:28:29.490670 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 708/1428] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:28:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:28:30.368466 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34051d47-4cf5-4078-bcba-7222d8050539 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:28:30.369021 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34051d47-4cf5-4078-bcba-7222d8050539 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:28:30.369021 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34051d47-4cf5-4078-bcba-7222d8050539 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:28:30.370455 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 721/1429] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 02:28:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:28:30.375354 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022a3d95-6431-4785-8e21-d85bd6ebc438 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:28:30.375794 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022a3d95-6431-4785-8e21-d85bd6ebc438 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:28:30.375938 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022a3d95-6431-4785-8e21-d85bd6ebc438 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:28:30.450452 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-022a3d95-6431-4785-8e21-d85bd6ebc438 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:28:30.451632 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-022a3d95-6431-4785-8e21-d85bd6ebc438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:28:30.546462 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 709/1430] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 02:28:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 02:28:30.697848 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34051d47-4cf5-4078-bcba-7222d8050539 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:28:30.699376 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-34051d47-4cf5-4078-bcba-7222d8050539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:28:30.747042 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:28:30.843285 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34051d47-4cf5-4078-bcba-7222d8050539 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:28:30.958544 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050df7a6-509d-4edf-9c01-86f26244d9d3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80b138831bc54c7b986d50092d62d4af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:28:30.959231 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050df7a6-509d-4edf-9c01-86f26244d9d3 service nova] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:28:30.959521 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050df7a6-509d-4edf-9c01-86f26244d9d3 service nova] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:28:30.959829 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050df7a6-509d-4edf-9c01-86f26244d9d3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:28:30.961571 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-050df7a6-509d-4edf-9c01-86f26244d9d3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:28:30.968626 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:28:31.052094 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050df7a6-509d-4edf-9c01-86f26244d9d3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:28:31.053687 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050df7a6-509d-4edf-9c01-86f26244d9d3 service nova] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:28:31.071210 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 722/1431] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:28:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:28:31.931460 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353b1d0f-739c-4b93-8e7e-41e58bc75db5 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:28:31.932001 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353b1d0f-739c-4b93-8e7e-41e58bc75db5 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:28:31.932001 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353b1d0f-739c-4b93-8e7e-41e58bc75db5 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:28:31.933407 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 710/1432] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 02:28:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:28:31.938533 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a576a4c2-267f-4909-804a-99b40354da29 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:28:31.938968 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a576a4c2-267f-4909-804a-99b40354da29 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:28:31.939076 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a576a4c2-267f-4909-804a-99b40354da29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:28:32.014270 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-a576a4c2-267f-4909-804a-99b40354da29 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:28:32.015500 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-a576a4c2-267f-4909-804a-99b40354da29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:28:32.083793 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:28:32.112267 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 723/1433] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 02:28:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:28:32.266862 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353b1d0f-739c-4b93-8e7e-41e58bc75db5 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:28:32.268286 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-353b1d0f-739c-4b93-8e7e-41e58bc75db5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:28:32.285506 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:28:32.413474 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353b1d0f-739c-4b93-8e7e-41e58bc75db5 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:28:32.500805 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:28:32.526857 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead5e019-535f-4700-9b70-3e87e5f44642 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80b138831bc54c7b986d50092d62d4af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:28:32.527434 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead5e019-535f-4700-9b70-3e87e5f44642 service nova] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:28:32.527715 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead5e019-535f-4700-9b70-3e87e5f44642 service nova] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:28:32.528108 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead5e019-535f-4700-9b70-3e87e5f44642 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:28:32.529651 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-ead5e019-535f-4700-9b70-3e87e5f44642 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:28:32.619807 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead5e019-535f-4700-9b70-3e87e5f44642 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:28:32.621427 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead5e019-535f-4700-9b70-3e87e5f44642 service nova] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:28:32.639330 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 711/1434] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:28:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:28:33.490040 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec3d4b9-76fc-4f7d-bfcb-98bdd9e59137 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:28:33.490649 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec3d4b9-76fc-4f7d-bfcb-98bdd9e59137 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:28:33.490649 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec3d4b9-76fc-4f7d-bfcb-98bdd9e59137 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:28:33.492124 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 724/1435] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 02:28:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:28:33.496948 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c1156e-4145-41c9-a341-271d1d4f505b None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:28:33.497368 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c1156e-4145-41c9-a341-271d1d4f505b None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:28:33.497571 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c1156e-4145-41c9-a341-271d1d4f505b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:28:33.572852 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-21c1156e-4145-41c9-a341-271d1d4f505b None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:28:33.574148 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-21c1156e-4145-41c9-a341-271d1d4f505b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:28:33.616613 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:28:33.671779 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 712/1436] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 02:28:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:28:33.823328 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec3d4b9-76fc-4f7d-bfcb-98bdd9e59137 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:28:33.824750 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-fec3d4b9-76fc-4f7d-bfcb-98bdd9e59137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:28:33.971012 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec3d4b9-76fc-4f7d-bfcb-98bdd9e59137 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:28:34.024373 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:28:34.083648 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a3b902-2ed6-42d5-b13d-be2da0b843cd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80b138831bc54c7b986d50092d62d4af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:28:34.084220 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a3b902-2ed6-42d5-b13d-be2da0b843cd service nova] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:28:34.084509 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a3b902-2ed6-42d5-b13d-be2da0b843cd service nova] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:28:34.084784 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a3b902-2ed6-42d5-b13d-be2da0b843cd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:28:34.086439 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-48a3b902-2ed6-42d5-b13d-be2da0b843cd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:28:34.174854 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a3b902-2ed6-42d5-b13d-be2da0b843cd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:28:34.176465 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a3b902-2ed6-42d5-b13d-be2da0b843cd service nova] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:28:34.195075 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 725/1437] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:28:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:28:35.039734 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daedfc29-39a6-466b-be05-d804ef68780a None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:28:35.040333 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daedfc29-39a6-466b-be05-d804ef68780a None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:28:35.040333 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daedfc29-39a6-466b-be05-d804ef68780a None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:28:35.041926 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 713/1438] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 02:28:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:28:35.046878 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f2f341-8a71-4b2c-b6a4-269934236865 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:28:35.047295 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f2f341-8a71-4b2c-b6a4-269934236865 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:28:35.047433 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f2f341-8a71-4b2c-b6a4-269934236865 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:28:35.132867 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-21f2f341-8a71-4b2c-b6a4-269934236865 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:28:35.134075 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-21f2f341-8a71-4b2c-b6a4-269934236865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:28:35.136130 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:28:35.229287 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 726/1439] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 02:28:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:28:35.379882 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daedfc29-39a6-466b-be05-d804ef68780a None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:28:35.381290 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-daedfc29-39a6-466b-be05-d804ef68780a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:28:35.527584 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daedfc29-39a6-466b-be05-d804ef68780a None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:28:35.555728 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:28:35.642808 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d535756-193f-46ea-bd21-d0632114d4b0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80b138831bc54c7b986d50092d62d4af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:28:35.643452 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d535756-193f-46ea-bd21-d0632114d4b0 service nova] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:28:35.643738 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d535756-193f-46ea-bd21-d0632114d4b0 service nova] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:28:35.644011 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d535756-193f-46ea-bd21-d0632114d4b0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:28:35.645647 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-6d535756-193f-46ea-bd21-d0632114d4b0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:28:35.734951 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d535756-193f-46ea-bd21-d0632114d4b0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:28:35.736540 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d535756-193f-46ea-bd21-d0632114d4b0 service nova] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:28:35.754854 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 714/1440] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:28:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:28:37.691409 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69fdc15-d234-4f45-aaa3-686fc550d6a2 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:28:37.691998 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69fdc15-d234-4f45-aaa3-686fc550d6a2 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:28:37.691998 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69fdc15-d234-4f45-aaa3-686fc550d6a2 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:28:37.693342 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 727/1441] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 02:28:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:28:37.698729 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fb5a6b-a174-4f86-b176-f4ed0f7032a2 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:28:37.699168 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fb5a6b-a174-4f86-b176-f4ed0f7032a2 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:28:37.699277 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fb5a6b-a174-4f86-b176-f4ed0f7032a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:28:37.738883 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:28:37.762541 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-51fb5a6b-a174-4f86-b176-f4ed0f7032a2 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:28:37.763734 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-51fb5a6b-a174-4f86-b176-f4ed0f7032a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:28:37.860638 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 715/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 02:28:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:28:37.899643 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69fdc15-d234-4f45-aaa3-686fc550d6a2 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:28:37.901143 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-b69fdc15-d234-4f45-aaa3-686fc550d6a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:28:38.003799 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69fdc15-d234-4f45-aaa3-686fc550d6a2 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:28:38.035996 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:28:38.077465 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18907d8-266e-4a77-aea0-a0d3295d3fc4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eaf19a259c2741cea375a38763e33cdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:28:38.078133 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18907d8-266e-4a77-aea0-a0d3295d3fc4 service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:28:38.078442 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18907d8-266e-4a77-aea0-a0d3295d3fc4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:28:38.078717 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18907d8-266e-4a77-aea0-a0d3295d3fc4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:28:38.080388 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-e18907d8-266e-4a77-aea0-a0d3295d3fc4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:28:38.187835 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18907d8-266e-4a77-aea0-a0d3295d3fc4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:28:38.189272 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18907d8-266e-4a77-aea0-a0d3295d3fc4 service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:28:38.207856 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 728/1443] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:28:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:28:40.371155 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad08ea58-faa3-4c63-beaa-c9e9fc4bfdc5 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:28:40.371720 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad08ea58-faa3-4c63-beaa-c9e9fc4bfdc5 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:28:40.371720 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad08ea58-faa3-4c63-beaa-c9e9fc4bfdc5 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:28:40.373090 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 716/1444] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 02:28:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:28:40.379340 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ce11c2-a96a-49c1-830e-b996166e1520 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:28:40.379807 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ce11c2-a96a-49c1-830e-b996166e1520 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:28:40.379916 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ce11c2-a96a-49c1-830e-b996166e1520 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:28:40.445601 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-a3ce11c2-a96a-49c1-830e-b996166e1520 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:28:40.446854 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-a3ce11c2-a96a-49c1-830e-b996166e1520 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:28:40.459578 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:28:40.544163 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 729/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 02:28:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:28:40.583784 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad08ea58-faa3-4c63-beaa-c9e9fc4bfdc5 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:28:40.585282 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-ad08ea58-faa3-4c63-beaa-c9e9fc4bfdc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:28:40.688241 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad08ea58-faa3-4c63-beaa-c9e9fc4bfdc5 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:28:40.700725 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:28:40.761787 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaea8d5-79a5-4b2c-8caf-adf7d24d5a89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eaf19a259c2741cea375a38763e33cdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:28:40.762488 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaea8d5-79a5-4b2c-8caf-adf7d24d5a89 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:28:40.762772 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaea8d5-79a5-4b2c-8caf-adf7d24d5a89 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:28:40.763049 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaea8d5-79a5-4b2c-8caf-adf7d24d5a89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:28:40.764695 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-ecaea8d5-79a5-4b2c-8caf-adf7d24d5a89 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:28:40.876401 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecaea8d5-79a5-4b2c-8caf-adf7d24d5a89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:28:40.877882 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecaea8d5-79a5-4b2c-8caf-adf7d24d5a89 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:28:40.896492 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 717/1446] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:28:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:28:43.248585 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9d19ec-ed50-4e05-89c9-df840b62fe8a None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:28:43.249164 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9d19ec-ed50-4e05-89c9-df840b62fe8a None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:28:43.249164 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9d19ec-ed50-4e05-89c9-df840b62fe8a None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:28:43.250563 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 730/1447] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 02:28:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:28:43.257105 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e947d4a4-8d83-4c18-b056-87fd43025166 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:28:43.257578 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e947d4a4-8d83-4c18-b056-87fd43025166 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:28:43.257685 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e947d4a4-8d83-4c18-b056-87fd43025166 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:28:43.304733 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:28:43.323792 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-e947d4a4-8d83-4c18-b056-87fd43025166 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:28:43.325042 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-e947d4a4-8d83-4c18-b056-87fd43025166 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:28:43.423362 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 718/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 02:28:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:28:43.464482 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9d19ec-ed50-4e05-89c9-df840b62fe8a None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:28:43.466022 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-ad9d19ec-ed50-4e05-89c9-df840b62fe8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:28:43.568550 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9d19ec-ed50-4e05-89c9-df840b62fe8a None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:28:43.608584 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:28:43.643214 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf56829d-3193-42a9-b590-2d11b3e12bd7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eaf19a259c2741cea375a38763e33cdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:28:43.643868 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf56829d-3193-42a9-b590-2d11b3e12bd7 service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:28:43.644152 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf56829d-3193-42a9-b590-2d11b3e12bd7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:28:43.644424 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf56829d-3193-42a9-b590-2d11b3e12bd7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:28:43.646073 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-bf56829d-3193-42a9-b590-2d11b3e12bd7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:28:43.752750 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf56829d-3193-42a9-b590-2d11b3e12bd7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:28:43.754217 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf56829d-3193-42a9-b590-2d11b3e12bd7 service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:28:43.773479 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 731/1449] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:28:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:28:45.907869 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eeb532-e4f1-4b08-960b-f5d0b706d24c None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:28:45.908685 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eeb532-e4f1-4b08-960b-f5d0b706d24c None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:28:45.908685 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eeb532-e4f1-4b08-960b-f5d0b706d24c None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:28:45.909810 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 719/1450] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 02:28:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:28:45.915255 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de747afe-16fd-4428-a204-5c0c70d25cc5 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:28:45.915716 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de747afe-16fd-4428-a204-5c0c70d25cc5 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:28:45.915824 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de747afe-16fd-4428-a204-5c0c70d25cc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:28:45.979923 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-de747afe-16fd-4428-a204-5c0c70d25cc5 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:28:45.981207 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-de747afe-16fd-4428-a204-5c0c70d25cc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:28:46.002663 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:28:46.076677 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 732/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 02:28:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:28:46.115760 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03eeb532-e4f1-4b08-960b-f5d0b706d24c None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:28:46.117288 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-03eeb532-e4f1-4b08-960b-f5d0b706d24c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:28:46.219213 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03eeb532-e4f1-4b08-960b-f5d0b706d24c None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:28:46.239706 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:28:46.293406 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b48e2b2-54ac-4843-9fe4-f09d3b46f943 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eaf19a259c2741cea375a38763e33cdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:28:46.294052 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b48e2b2-54ac-4843-9fe4-f09d3b46f943 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:28:46.294363 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b48e2b2-54ac-4843-9fe4-f09d3b46f943 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:28:46.294639 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b48e2b2-54ac-4843-9fe4-f09d3b46f943 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:28:46.296260 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-2b48e2b2-54ac-4843-9fe4-f09d3b46f943 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:28:46.402597 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b48e2b2-54ac-4843-9fe4-f09d3b46f943 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:28:46.404041 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b48e2b2-54ac-4843-9fe4-f09d3b46f943 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:28:46.422637 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 720/1452] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:28:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:28:48.555778 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc56056-74bd-4f6d-bed3-e6f50f4853ba None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:28:48.556353 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc56056-74bd-4f6d-bed3-e6f50f4853ba None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:28:48.556353 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc56056-74bd-4f6d-bed3-e6f50f4853ba None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:28:48.557747 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 733/1453] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 02:28:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:28:48.563237 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7d458f-f07c-4b9a-b87d-7f58de9c17aa None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:28:48.563706 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7d458f-f07c-4b9a-b87d-7f58de9c17aa None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:28:48.563818 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7d458f-f07c-4b9a-b87d-7f58de9c17aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:28:48.630038 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:28:48.632953 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-8a7d458f-f07c-4b9a-b87d-7f58de9c17aa None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:28:48.634292 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-8a7d458f-f07c-4b9a-b87d-7f58de9c17aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:28:48.731111 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 721/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 02:28:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:28:48.768997 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc56056-74bd-4f6d-bed3-e6f50f4853ba None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:28:48.770593 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-5cc56056-74bd-4f6d-bed3-e6f50f4853ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:28:48.873503 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc56056-74bd-4f6d-bed3-e6f50f4853ba None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:28:48.921890 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:28:48.947485 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fd5c9a-1783-4a56-a6b3-92a9d4c09a21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eaf19a259c2741cea375a38763e33cdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:28:48.948199 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fd5c9a-1783-4a56-a6b3-92a9d4c09a21 service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:28:48.948485 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fd5c9a-1783-4a56-a6b3-92a9d4c09a21 service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:28:48.948760 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fd5c9a-1783-4a56-a6b3-92a9d4c09a21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:28:48.950417 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-d2fd5c9a-1783-4a56-a6b3-92a9d4c09a21 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:28:49.060894 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fd5c9a-1783-4a56-a6b3-92a9d4c09a21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:28:49.062346 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fd5c9a-1783-4a56-a6b3-92a9d4c09a21 service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:28:49.080497 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 734/1455] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:28:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:28:51.349997 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a77c2e2-1c6b-4638-9077-2b7abc019978 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:28:51.350590 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a77c2e2-1c6b-4638-9077-2b7abc019978 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:28:51.350590 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a77c2e2-1c6b-4638-9077-2b7abc019978 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:28:51.351992 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 722/1456] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 02:28:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:28:51.357230 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578fa269-3485-4c58-b6c4-ebb257db5caf None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:28:51.357680 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578fa269-3485-4c58-b6c4-ebb257db5caf None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:28:51.357792 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578fa269-3485-4c58-b6c4-ebb257db5caf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:28:51.422508 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-578fa269-3485-4c58-b6c4-ebb257db5caf None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:28:51.423843 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-578fa269-3485-4c58-b6c4-ebb257db5caf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:28:51.453629 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:28:51.522287 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 735/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 02:28:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:28:51.561538 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a77c2e2-1c6b-4638-9077-2b7abc019978 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:28:51.563072 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-1a77c2e2-1c6b-4638-9077-2b7abc019978 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:28:51.671693 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a77c2e2-1c6b-4638-9077-2b7abc019978 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:28:51.700535 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:28:51.745853 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b5412b-41d1-463b-bd1c-657f0cd4e99c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eaf19a259c2741cea375a38763e33cdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:28:51.746541 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b5412b-41d1-463b-bd1c-657f0cd4e99c service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:28:51.746829 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b5412b-41d1-463b-bd1c-657f0cd4e99c service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:28:51.747109 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b5412b-41d1-463b-bd1c-657f0cd4e99c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:28:51.748753 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-72b5412b-41d1-463b-bd1c-657f0cd4e99c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:28:51.857157 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b5412b-41d1-463b-bd1c-657f0cd4e99c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:28:51.858611 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b5412b-41d1-463b-bd1c-657f0cd4e99c service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:28:51.876860 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 723/1458] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:28:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:28:53.930614 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9053802-ca7c-479b-9ca3-d3740669a52d None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:28:53.931173 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9053802-ca7c-479b-9ca3-d3740669a52d None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:28:53.931173 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9053802-ca7c-479b-9ca3-d3740669a52d None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:28:53.932558 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 736/1459] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 02:28:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:28:53.937781 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57d611b-1f6a-42be-9cc5-57ba7f942378 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:28:53.938239 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57d611b-1f6a-42be-9cc5-57ba7f942378 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:28:53.938403 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57d611b-1f6a-42be-9cc5-57ba7f942378 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:28:54.001815 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-d57d611b-1f6a-42be-9cc5-57ba7f942378 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:28:54.003041 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-d57d611b-1f6a-42be-9cc5-57ba7f942378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:28:54.011193 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:28:54.099330 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 724/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 02:28:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:28:54.135832 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9053802-ca7c-479b-9ca3-d3740669a52d None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:28:54.137318 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-d9053802-ca7c-479b-9ca3-d3740669a52d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:28:54.240411 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9053802-ca7c-479b-9ca3-d3740669a52d None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:28:54.295293 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:28:54.314019 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd82952-6c1d-4317-b8e3-e3f06f7f995c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eaf19a259c2741cea375a38763e33cdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:28:54.314697 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd82952-6c1d-4317-b8e3-e3f06f7f995c service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:28:54.314981 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd82952-6c1d-4317-b8e3-e3f06f7f995c service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:28:54.315258 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd82952-6c1d-4317-b8e3-e3f06f7f995c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:28:54.316893 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-0bd82952-6c1d-4317-b8e3-e3f06f7f995c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:28:54.424266 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd82952-6c1d-4317-b8e3-e3f06f7f995c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:28:54.425705 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd82952-6c1d-4317-b8e3-e3f06f7f995c service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:28:54.444424 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 737/1461] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:28:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:28:56.504284 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df0d81f-ab16-4cc4-96f3-ca7bc7b380c2 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:28:56.504896 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df0d81f-ab16-4cc4-96f3-ca7bc7b380c2 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:28:56.504896 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df0d81f-ab16-4cc4-96f3-ca7bc7b380c2 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:28:56.506243 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 725/1462] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 02:28:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:28:56.511490 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694aea1f-928e-43c2-b02c-652414e16c6c None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:28:56.511915 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694aea1f-928e-43c2-b02c-652414e16c6c None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:28:56.512024 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694aea1f-928e-43c2-b02c-652414e16c6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:28:56.576907 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-694aea1f-928e-43c2-b02c-652414e16c6c None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:28:56.585953 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-694aea1f-928e-43c2-b02c-652414e16c6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:28:56.616728 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:28:56.675268 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 738/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 02:28:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:28:56.713477 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df0d81f-ab16-4cc4-96f3-ca7bc7b380c2 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:28:56.715018 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-7df0d81f-ab16-4cc4-96f3-ca7bc7b380c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:28:56.818109 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df0d81f-ab16-4cc4-96f3-ca7bc7b380c2 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:28:56.854923 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:28:56.889809 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c9e5c1-74a1-423c-aac7-271efa237358 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eaf19a259c2741cea375a38763e33cdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:28:56.890534 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c9e5c1-74a1-423c-aac7-271efa237358 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:28:56.891011 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c9e5c1-74a1-423c-aac7-271efa237358 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:28:56.892030 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c9e5c1-74a1-423c-aac7-271efa237358 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:28:56.894030 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-86c9e5c1-74a1-423c-aac7-271efa237358 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:28:57.000602 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c9e5c1-74a1-423c-aac7-271efa237358 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:28:57.002671 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c9e5c1-74a1-423c-aac7-271efa237358 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:28:57.020761 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 726/1464] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:28:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:28:59.045924 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66edc1ab-0600-4f0b-a260-e7b971cfe445 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:28:59.046535 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66edc1ab-0600-4f0b-a260-e7b971cfe445 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:28:59.046535 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66edc1ab-0600-4f0b-a260-e7b971cfe445 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:28:59.048066 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 739/1465] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 02:28:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:28:59.053275 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acff4591-8302-47e1-81ef-eb3a20857d9d None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:28:59.053688 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acff4591-8302-47e1-81ef-eb3a20857d9d None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:28:59.053834 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acff4591-8302-47e1-81ef-eb3a20857d9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:28:59.117233 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-acff4591-8302-47e1-81ef-eb3a20857d9d None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:28:59.118455 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-acff4591-8302-47e1-81ef-eb3a20857d9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:28:59.136808 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:28:59.217110 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 727/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 02:28:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:28:59.254456 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66edc1ab-0600-4f0b-a260-e7b971cfe445 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:28:59.255964 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-66edc1ab-0600-4f0b-a260-e7b971cfe445 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:28:59.356854 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66edc1ab-0600-4f0b-a260-e7b971cfe445 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:28:59.425795 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:28:59.433123 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00083f4-48fa-45c1-8032-d1ceb5eb2207 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eaf19a259c2741cea375a38763e33cdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:28:59.433771 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00083f4-48fa-45c1-8032-d1ceb5eb2207 service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:28:59.434059 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00083f4-48fa-45c1-8032-d1ceb5eb2207 service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:28:59.434378 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00083f4-48fa-45c1-8032-d1ceb5eb2207 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:28:59.436027 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-e00083f4-48fa-45c1-8032-d1ceb5eb2207 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:28:59.548929 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00083f4-48fa-45c1-8032-d1ceb5eb2207 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:28:59.550607 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00083f4-48fa-45c1-8032-d1ceb5eb2207 service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:28:59.568703 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 740/1467] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:28:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:29:01.594224 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1510d7-5da6-499f-8204-a54805f29a55 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:29:01.594690 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1510d7-5da6-499f-8204-a54805f29a55 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:29:01.594939 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1510d7-5da6-499f-8204-a54805f29a55 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:29:01.596359 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 728/1468] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 02:29:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:29:01.601919 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a497a82-fd12-486a-9652-eaeaa23cc030 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:29:01.602667 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a497a82-fd12-486a-9652-eaeaa23cc030 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:29:01.602667 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a497a82-fd12-486a-9652-eaeaa23cc030 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:29:01.666884 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-6a497a82-fd12-486a-9652-eaeaa23cc030 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:29:01.668102 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-6a497a82-fd12-486a-9652-eaeaa23cc030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:29:01.715944 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:29:01.764345 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 741/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 02:29:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:29:01.802177 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1510d7-5da6-499f-8204-a54805f29a55 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:29:01.803723 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-9e1510d7-5da6-499f-8204-a54805f29a55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:29:01.907995 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1510d7-5da6-499f-8204-a54805f29a55 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:29:01.956542 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:29:01.982513 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb282d0-82d8-4887-adb1-956f1f5a198f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eaf19a259c2741cea375a38763e33cdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:29:01.983368 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb282d0-82d8-4887-adb1-956f1f5a198f service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:29:01.983888 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb282d0-82d8-4887-adb1-956f1f5a198f service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:29:01.984165 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb282d0-82d8-4887-adb1-956f1f5a198f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:29:01.986755 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-9bb282d0-82d8-4887-adb1-956f1f5a198f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:29:02.093173 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb282d0-82d8-4887-adb1-956f1f5a198f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:29:02.095457 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb282d0-82d8-4887-adb1-956f1f5a198f service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:29:02.114249 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 729/1470] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:29:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:29:11.762496 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=22 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:29:12.111761 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=66 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:29:21.802317 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34606c6-33f2-4503-a1dd-a8290cb97dec None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:29:21.802895 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34606c6-33f2-4503-a1dd-a8290cb97dec None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:29:21.802895 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34606c6-33f2-4503-a1dd-a8290cb97dec None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:29:21.804311 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 742/1471] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:29:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:29:21.809500 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826d0cf3-6528-4bbe-acca-714a5944cbb0 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:29:21.809933 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826d0cf3-6528-4bbe-acca-714a5944cbb0 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:29:21.810061 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826d0cf3-6528-4bbe-acca-714a5944cbb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:29:21.874912 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-826d0cf3-6528-4bbe-acca-714a5944cbb0 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:29:21.876066 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-826d0cf3-6528-4bbe-acca-714a5944cbb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:29:21.972308 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 730/1472] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:29:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:29:22.020303 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab381e36-894f-4861-9531-5ff2e292b789 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:29:22.020590 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab381e36-894f-4861-9531-5ff2e292b789 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:29:22.020865 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab381e36-894f-4861-9531-5ff2e292b789 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:29:22.022305 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 743/1473] 10.222.0.66 () {54 vars in 961 bytes} [Mon Jul 18 02:29:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:29:22.027020 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4846e6-f671-432c-9448-10c1125dfab5 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:29:22.027322 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4846e6-f671-432c-9448-10c1125dfab5 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:29:22.027604 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4846e6-f671-432c-9448-10c1125dfab5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:29:22.101025 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-4d4846e6-f671-432c-9448-10c1125dfab5 None None] MFA Rules not processed for user `e1391b81e7234bc18e22b9e84f231256`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:29:22.102258 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-4d4846e6-f671-432c-9448-10c1125dfab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:29:22.122558 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:29:22.178527 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 731/1474] 10.222.0.66 () {58 vars in 1058 bytes} [Mon Jul 18 02:29:22 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:29:22.186246 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab381e36-894f-4861-9531-5ff2e292b789 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:29:22.187885 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-ab381e36-894f-4861-9531-5ff2e292b789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:29:22.289445 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab381e36-894f-4861-9531-5ff2e292b789 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:29:22.365830 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca259cf-67c6-4ddf-8bf3-aeafadf6964e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1391b81e7234bc18e22b9e84f231256', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:29:22.366506 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca259cf-67c6-4ddf-8bf3-aeafadf6964e service glance] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:29:22.366797 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca259cf-67c6-4ddf-8bf3-aeafadf6964e service glance] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:29:22.367071 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca259cf-67c6-4ddf-8bf3-aeafadf6964e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:29:22.368716 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-cca259cf-67c6-4ddf-8bf3-aeafadf6964e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:29:22.423690 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:29:22.478242 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca259cf-67c6-4ddf-8bf3-aeafadf6964e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:29:22.479611 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca259cf-67c6-4ddf-8bf3-aeafadf6964e service glance] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:29:22.497115 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 744/1475] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:29:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:29:24.605937 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4880da9f-ffe7-45fe-9c2d-17e0cd6777c2 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:29:24.606541 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4880da9f-ffe7-45fe-9c2d-17e0cd6777c2 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:29:24.606541 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4880da9f-ffe7-45fe-9c2d-17e0cd6777c2 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:29:24.608061 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 732/1476] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:29:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:29:24.613622 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121a1235-f45a-4bec-96ee-508978f38266 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:29:24.614069 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121a1235-f45a-4bec-96ee-508978f38266 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:29:24.614215 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121a1235-f45a-4bec-96ee-508978f38266 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:29:24.679971 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-121a1235-f45a-4bec-96ee-508978f38266 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:29:24.681212 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-121a1235-f45a-4bec-96ee-508978f38266 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:29:24.777249 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 745/1477] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:29:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 02:29:24.801290 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4880da9f-ffe7-45fe-9c2d-17e0cd6777c2 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:29:24.802845 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-4880da9f-ffe7-45fe-9c2d-17e0cd6777c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:29:24.928233 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4880da9f-ffe7-45fe-9c2d-17e0cd6777c2 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:29:24.976695 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:29:25.021721 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c434ce5-0114-40a4-92e7-09cb86498acb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:29:25.022410 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c434ce5-0114-40a4-92e7-09cb86498acb admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:29:25.022696 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c434ce5-0114-40a4-92e7-09cb86498acb admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:29:25.022972 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c434ce5-0114-40a4-92e7-09cb86498acb admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:29:25.023695 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c434ce5-0114-40a4-92e7-09cb86498acb admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:29:25.025444 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: WARNING py.warnings [None req-1c434ce5-0114-40a4-92e7-09cb86498acb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 02:29:25.025444 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: warnings.warn(msg) Jul 18 02:29:25.025444 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: Jul 18 02:29:25.026624 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c434ce5-0114-40a4-92e7-09cb86498acb admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:29:25.046515 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 733/1478] 10.222.0.66 () {56 vars in 1108 bytes} [Mon Jul 18 02:29:24 2022] GET /identity/v3/projects => generated 2553 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:29:27.081930 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9091b0a-0b54-4b79-9c24-b4881072d712 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:29:27.082542 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9091b0a-0b54-4b79-9c24-b4881072d712 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:29:27.082542 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9091b0a-0b54-4b79-9c24-b4881072d712 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:29:27.083930 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 746/1479] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:29:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:29:27.089163 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f629b32-ba50-48a8-b95a-ed19a09110b6 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:29:27.089569 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f629b32-ba50-48a8-b95a-ed19a09110b6 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:29:27.089722 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f629b32-ba50-48a8-b95a-ed19a09110b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:29:27.153627 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-1f629b32-ba50-48a8-b95a-ed19a09110b6 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:29:27.154892 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-1f629b32-ba50-48a8-b95a-ed19a09110b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:29:27.250611 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 734/1480] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:29:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:29:27.297721 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9091b0a-0b54-4b79-9c24-b4881072d712 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:29:27.299268 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-e9091b0a-0b54-4b79-9c24-b4881072d712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:29:27.308947 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:29:27.444460 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9091b0a-0b54-4b79-9c24-b4881072d712 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:29:27.523377 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:29:27.557440 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006a03e3-4364-4a46-8026-998408973607 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80b138831bc54c7b986d50092d62d4af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:29:27.558114 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006a03e3-4364-4a46-8026-998408973607 service nova] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:29:27.558410 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006a03e3-4364-4a46-8026-998408973607 service nova] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:29:27.558687 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006a03e3-4364-4a46-8026-998408973607 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:29:27.560340 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-006a03e3-4364-4a46-8026-998408973607 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:29:27.652585 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006a03e3-4364-4a46-8026-998408973607 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:29:27.654776 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006a03e3-4364-4a46-8026-998408973607 service nova] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:29:27.673385 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 747/1481] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:29:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:29:29.856820 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b3212b-bf98-4f01-bf85-5dc9a14ede63 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:29:29.857389 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b3212b-bf98-4f01-bf85-5dc9a14ede63 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:29:29.857389 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b3212b-bf98-4f01-bf85-5dc9a14ede63 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:29:29.858970 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 735/1482] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:29:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:29:29.864479 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b041da-f8be-48c8-8e1d-57317e6a464a None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:29:29.864907 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b041da-f8be-48c8-8e1d-57317e6a464a None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:29:29.865040 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b041da-f8be-48c8-8e1d-57317e6a464a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:29:30.048331 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-84b041da-f8be-48c8-8e1d-57317e6a464a None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:29:30.049545 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-84b041da-f8be-48c8-8e1d-57317e6a464a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:29:30.094933 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:29:30.155100 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 748/1483] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:29:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:29:30.191976 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b3212b-bf98-4f01-bf85-5dc9a14ede63 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:29:30.193496 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-30b3212b-bf98-4f01-bf85-5dc9a14ede63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:29:30.216566 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:29:30.298001 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b3212b-bf98-4f01-bf85-5dc9a14ede63 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:29:30.370888 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570b2e81-1886-40c2-8961-ef86894a7bb2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eaf19a259c2741cea375a38763e33cdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:29:30.371529 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570b2e81-1886-40c2-8961-ef86894a7bb2 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:29:30.371811 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570b2e81-1886-40c2-8961-ef86894a7bb2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:29:30.372080 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570b2e81-1886-40c2-8961-ef86894a7bb2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:29:30.373886 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-570b2e81-1886-40c2-8961-ef86894a7bb2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:29:30.451566 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:29:30.486850 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570b2e81-1886-40c2-8961-ef86894a7bb2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:29:30.488295 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570b2e81-1886-40c2-8961-ef86894a7bb2 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:29:30.506039 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 736/1484] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:29:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:29:32.569745 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba59e25-c94e-4cd6-95bf-3678d6a508f1 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:29:32.570345 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba59e25-c94e-4cd6-95bf-3678d6a508f1 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:29:32.570345 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba59e25-c94e-4cd6-95bf-3678d6a508f1 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:29:32.571707 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 749/1485] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:29:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:29:32.577109 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c15033a-c0f9-4e27-a238-c6c0842437c5 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:29:32.577547 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c15033a-c0f9-4e27-a238-c6c0842437c5 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:29:32.577647 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c15033a-c0f9-4e27-a238-c6c0842437c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:29:32.643313 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-6c15033a-c0f9-4e27-a238-c6c0842437c5 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:29:32.644511 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-6c15033a-c0f9-4e27-a238-c6c0842437c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:29:32.747556 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 737/1486] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:29:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:29:32.785729 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba59e25-c94e-4cd6-95bf-3678d6a508f1 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:29:32.787274 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-3ba59e25-c94e-4cd6-95bf-3678d6a508f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:29:32.888738 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba59e25-c94e-4cd6-95bf-3678d6a508f1 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:29:32.960997 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:29:32.962828 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa87acb0-0acb-4ce3-9103-508b6d2cdee0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eaf19a259c2741cea375a38763e33cdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:29:32.963512 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa87acb0-0acb-4ce3-9103-508b6d2cdee0 service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:29:32.963796 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa87acb0-0acb-4ce3-9103-508b6d2cdee0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:29:32.964073 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa87acb0-0acb-4ce3-9103-508b6d2cdee0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:29:32.965745 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-fa87acb0-0acb-4ce3-9103-508b6d2cdee0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:29:33.073721 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa87acb0-0acb-4ce3-9103-508b6d2cdee0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:29:33.075112 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa87acb0-0acb-4ce3-9103-508b6d2cdee0 service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:29:33.095158 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 750/1487] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:29:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:29:35.218567 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421bb6b2-4c1f-4996-afd7-00da3d87ae6e None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:29:35.219130 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421bb6b2-4c1f-4996-afd7-00da3d87ae6e None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:29:35.219130 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421bb6b2-4c1f-4996-afd7-00da3d87ae6e None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:29:35.220522 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 738/1488] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:29:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:29:35.225930 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eab7123-3125-4655-81dd-9c5249b7fea1 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:29:35.226394 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eab7123-3125-4655-81dd-9c5249b7fea1 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:29:35.226556 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eab7123-3125-4655-81dd-9c5249b7fea1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:29:35.292259 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-6eab7123-3125-4655-81dd-9c5249b7fea1 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:29:35.293462 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-6eab7123-3125-4655-81dd-9c5249b7fea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:29:35.345043 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:29:35.390322 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 751/1489] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:29:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:29:35.430687 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421bb6b2-4c1f-4996-afd7-00da3d87ae6e None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:29:35.432219 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-421bb6b2-4c1f-4996-afd7-00da3d87ae6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:29:35.468985 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:29:35.536486 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421bb6b2-4c1f-4996-afd7-00da3d87ae6e None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:29:35.609619 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581b8a92-bf92-4e65-a076-326326cd4d73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eaf19a259c2741cea375a38763e33cdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:29:35.610274 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581b8a92-bf92-4e65-a076-326326cd4d73 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:29:35.610549 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581b8a92-bf92-4e65-a076-326326cd4d73 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:29:35.610824 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581b8a92-bf92-4e65-a076-326326cd4d73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:29:35.612524 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-581b8a92-bf92-4e65-a076-326326cd4d73 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:29:35.693368 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:29:35.721713 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581b8a92-bf92-4e65-a076-326326cd4d73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:29:35.723181 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581b8a92-bf92-4e65-a076-326326cd4d73 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:29:35.742384 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 739/1490] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:29:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:29:38.074592 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201617a3-f3cb-48dd-9dc5-0f27f479e048 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:29:38.075192 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201617a3-f3cb-48dd-9dc5-0f27f479e048 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:29:38.075192 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201617a3-f3cb-48dd-9dc5-0f27f479e048 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:29:38.077307 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 752/1491] 10.222.0.66 () {54 vars in 926 bytes} [Mon Jul 18 02:29:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:29:38.082517 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afc6ca2-9cd2-472b-ad96-57884835317a None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:29:38.082939 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afc6ca2-9cd2-472b-ad96-57884835317a None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:29:38.083064 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afc6ca2-9cd2-472b-ad96-57884835317a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:29:38.146049 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-1afc6ca2-9cd2-472b-ad96-57884835317a None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:29:38.147274 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-1afc6ca2-9cd2-472b-ad96-57884835317a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:29:38.241918 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 740/1492] 10.222.0.66 () {58 vars in 1023 bytes} [Mon Jul 18 02:29:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:29:38.280896 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201617a3-f3cb-48dd-9dc5-0f27f479e048 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:29:38.282565 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-201617a3-f3cb-48dd-9dc5-0f27f479e048 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:29:38.394577 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201617a3-f3cb-48dd-9dc5-0f27f479e048 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:29:38.467473 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3303ff2d-13bf-4fcd-b55a-dfc85eae1c5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eaf19a259c2741cea375a38763e33cdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:29:38.468122 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3303ff2d-13bf-4fcd-b55a-dfc85eae1c5f service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:29:38.468404 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3303ff2d-13bf-4fcd-b55a-dfc85eae1c5f service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:29:38.468680 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3303ff2d-13bf-4fcd-b55a-dfc85eae1c5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:29:38.470341 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-3303ff2d-13bf-4fcd-b55a-dfc85eae1c5f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:29:38.479286 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:29:38.579166 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3303ff2d-13bf-4fcd-b55a-dfc85eae1c5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:29:38.580617 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3303ff2d-13bf-4fcd-b55a-dfc85eae1c5f service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:29:38.599384 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 753/1493] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:29:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:29:48.240152 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=82 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:29:48.596962 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=50 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:31:13.691242 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fc0ded-8b33-4df9-9d0a-05fb6915f021 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:31:13.692444 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fc0ded-8b33-4df9-9d0a-05fb6915f021 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:31:13.692444 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fc0ded-8b33-4df9-9d0a-05fb6915f021 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:31:13.765387 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-40fc0ded-8b33-4df9-9d0a-05fb6915f021 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:31:13.766631 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-40fc0ded-8b33-4df9-9d0a-05fb6915f021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:31:13.868105 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 741/1494] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:31:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:31:13.874642 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30db7d9a-37f2-4624-9932-7cc010625c31 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:31:13.875112 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30db7d9a-37f2-4624-9932-7cc010625c31 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:31:13.875338 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30db7d9a-37f2-4624-9932-7cc010625c31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:31:13.953222 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-30db7d9a-37f2-4624-9932-7cc010625c31 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:31:13.954389 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-30db7d9a-37f2-4624-9932-7cc010625c31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:31:14.049383 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 754/1495] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:31:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:31:14.054860 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fc0ded-8b33-4df9-9d0a-05fb6915f021 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:31:14.056076 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-40fc0ded-8b33-4df9-9d0a-05fb6915f021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:31:14.068220 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fc0ded-8b33-4df9-9d0a-05fb6915f021 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:31:14.129900 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:31:14.169200 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78be173-1d53-4a6f-83b6-c89f2b9d365b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:31:14.169859 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78be173-1d53-4a6f-83b6-c89f2b9d365b admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:31:14.170183 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78be173-1d53-4a6f-83b6-c89f2b9d365b admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:31:14.170465 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78be173-1d53-4a6f-83b6-c89f2b9d365b admin admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:31:14.171445 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78be173-1d53-4a6f-83b6-c89f2b9d365b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:31:14.171760 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78be173-1d53-4a6f-83b6-c89f2b9d365b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:31:14.172786 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: WARNING py.warnings [None req-c78be173-1d53-4a6f-83b6-c89f2b9d365b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 18 02:31:14.172786 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: warnings.warn(msg) Jul 18 02:31:14.172786 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: Jul 18 02:31:14.173685 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78be173-1d53-4a6f-83b6-c89f2b9d365b admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:31:14.186280 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 742/1496] 10.222.0.66 () {58 vars in 1152 bytes} [Mon Jul 18 02:31:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:31:14.194584 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30db7d9a-37f2-4624-9932-7cc010625c31 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:31:14.203650 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30db7d9a-37f2-4624-9932-7cc010625c31 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:31:14.264292 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:31:14.299056 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28eb2c8-371d-4f98-a496-3be509b02931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:31:14.299746 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28eb2c8-371d-4f98-a496-3be509b02931 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:31:14.300030 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28eb2c8-371d-4f98-a496-3be509b02931 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:31:14.300304 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28eb2c8-371d-4f98-a496-3be509b02931 admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:31:14.301742 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28eb2c8-371d-4f98-a496-3be509b02931 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:31:14.303519 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: WARNING py.warnings [None req-f28eb2c8-371d-4f98-a496-3be509b02931 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 02:31:14.303519 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: warnings.warn(msg) Jul 18 02:31:14.303519 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: Jul 18 02:31:14.304726 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28eb2c8-371d-4f98-a496-3be509b02931 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:31:14.432706 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 755/1497] 10.222.0.66 () {60 vars in 1152 bytes} [Mon Jul 18 02:31:14 2022] POST /identity/v3/projects => generated 375 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:31:14.436874 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78be173-1d53-4a6f-83b6-c89f2b9d365b admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:31:14.438440 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-c78be173-1d53-4a6f-83b6-c89f2b9d365b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:31:14.471638 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78be173-1d53-4a6f-83b6-c89f2b9d365b admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:31:14.565475 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e15192-6cf1-480b-848d-7210946cf564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:31:14.566173 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e15192-6cf1-480b-848d-7210946cf564 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:31:14.566444 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e15192-6cf1-480b-848d-7210946cf564 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:31:14.566726 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e15192-6cf1-480b-848d-7210946cf564 admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:31:14.567409 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e15192-6cf1-480b-848d-7210946cf564 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:31:14.568425 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: WARNING py.warnings [None req-61e15192-6cf1-480b-848d-7210946cf564 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 02:31:14.568425 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: warnings.warn(msg) Jul 18 02:31:14.568425 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: Jul 18 02:31:14.569636 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e15192-6cf1-480b-848d-7210946cf564 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:31:14.700274 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 743/1498] 10.222.0.66 () {60 vars in 1143 bytes} [Mon Jul 18 02:31:14 2022] POST /identity/v3/users => generated 288 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:31:14.704585 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28eb2c8-371d-4f98-a496-3be509b02931 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:31:14.706103 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-f28eb2c8-371d-4f98-a496-3be509b02931 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:31:14.768300 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28eb2c8-371d-4f98-a496-3be509b02931 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:31:14.853315 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:31:14.863716 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f53780-d990-4853-8336-9c38109d6002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:31:14.866400 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f53780-d990-4853-8336-9c38109d6002 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:31:14.866400 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f53780-d990-4853-8336-9c38109d6002 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:31:14.866400 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f53780-d990-4853-8336-9c38109d6002 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:31:14.866400 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f53780-d990-4853-8336-9c38109d6002 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:31:14.866801 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: WARNING py.warnings [None req-a3f53780-d990-4853-8336-9c38109d6002 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 18 02:31:14.866801 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: warnings.warn(msg) Jul 18 02:31:14.866801 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: Jul 18 02:31:14.867497 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f53780-d990-4853-8336-9c38109d6002 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:31:14.880179 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 756/1499] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:31:14 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:31:14.884023 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e15192-6cf1-480b-848d-7210946cf564 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:31:14.885565 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-61e15192-6cf1-480b-848d-7210946cf564 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:31:14.898020 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e15192-6cf1-480b-848d-7210946cf564 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:31:14.923881 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:31:14.993407 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd2e39d-b188-459d-a1ce-c7b8f3e62b57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:31:14.994073 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd2e39d-b188-459d-a1ce-c7b8f3e62b57 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:31:14.994380 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd2e39d-b188-459d-a1ce-c7b8f3e62b57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:31:14.994651 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd2e39d-b188-459d-a1ce-c7b8f3e62b57 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:31:14.995366 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd2e39d-b188-459d-a1ce-c7b8f3e62b57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:31:14.996376 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: WARNING py.warnings [None req-6bd2e39d-b188-459d-a1ce-c7b8f3e62b57 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 18 02:31:14.996376 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: warnings.warn(msg) Jul 18 02:31:14.996376 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: Jul 18 02:31:14.997257 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd2e39d-b188-459d-a1ce-c7b8f3e62b57 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:31:15.010210 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 744/1500] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:31:14 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:31:15.014486 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f53780-d990-4853-8336-9c38109d6002 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:31:15.016154 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-a3f53780-d990-4853-8336-9c38109d6002 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:31:15.049562 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f53780-d990-4853-8336-9c38109d6002 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:31:15.146891 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf24bd8-6e8e-4f40-b1a1-1e02aae3f6f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:31:15.150971 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf24bd8-6e8e-4f40-b1a1-1e02aae3f6f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:31:15.150971 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf24bd8-6e8e-4f40-b1a1-1e02aae3f6f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:31:15.150971 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf24bd8-6e8e-4f40-b1a1-1e02aae3f6f2 admin admin] PATH_INFO: `/v3/projects/4468af83d6fb446fa69cd11024d8c564/users/f140d30f190f447580ea80d59a082241/roles/65feabf3b27944a8b01a86c8ac608457` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:31:15.170839 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf24bd8-6e8e-4f40-b1a1-1e02aae3f6f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4468af83d6fb446fa69cd11024d8c564, user_id=f140d30f190f447580ea80d59a082241, role_id=65feabf3b27944a8b01a86c8ac608457)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:31:15.173971 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: WARNING py.warnings [None req-cbf24bd8-6e8e-4f40-b1a1-1e02aae3f6f2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 02:31:15.173971 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: warnings.warn(msg) Jul 18 02:31:15.173971 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: Jul 18 02:31:15.179661 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf24bd8-6e8e-4f40-b1a1-1e02aae3f6f2 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:31:15.300208 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 757/1501] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:31:15 2022] PUT /identity/v3/projects/4468af83d6fb446fa69cd11024d8c564/users/f140d30f190f447580ea80d59a082241/roles/65feabf3b27944a8b01a86c8ac608457 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:31:15.312980 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10936eac-0c2c-4011-ae79-8aa5f407dbaf None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:31:15.313277 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10936eac-0c2c-4011-ae79-8aa5f407dbaf None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:31:15.313557 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10936eac-0c2c-4011-ae79-8aa5f407dbaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:31:15.368073 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-10936eac-0c2c-4011-ae79-8aa5f407dbaf None None] MFA Rules not processed for user `f140d30f190f447580ea80d59a082241`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:31:15.369269 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-10936eac-0c2c-4011-ae79-8aa5f407dbaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:31:15.446458 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 745/1502] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:31:15 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:31:15.459612 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f95a63-3a03-4804-b1c2-1f7311efa727 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:31:15.459903 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f95a63-3a03-4804-b1c2-1f7311efa727 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:31:15.460214 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f95a63-3a03-4804-b1c2-1f7311efa727 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:31:15.461586 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 758/1503] 10.222.0.66 () {54 vars in 964 bytes} [Mon Jul 18 02:31:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:31:15.466603 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d730717-bb37-428b-94b6-bed8e15e935b None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:31:15.466898 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d730717-bb37-428b-94b6-bed8e15e935b None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:31:15.467178 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d730717-bb37-428b-94b6-bed8e15e935b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:31:15.498244 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:31:15.542964 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-7d730717-bb37-428b-94b6-bed8e15e935b None None] MFA Rules not processed for user `cd49388facdf446885e55b45ef91f01e`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:31:15.544171 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-7d730717-bb37-428b-94b6-bed8e15e935b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:31:15.620227 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 746/1504] 10.222.0.66 () {58 vars in 1061 bytes} [Mon Jul 18 02:31:15 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:31:15.627868 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f95a63-3a03-4804-b1c2-1f7311efa727 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:31:15.629962 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-84f95a63-3a03-4804-b1c2-1f7311efa727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:31:15.685708 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:31:15.738773 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f95a63-3a03-4804-b1c2-1f7311efa727 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:31:15.812591 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c76d264-7bb2-4ded-8a2b-10debc6cf39b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd49388facdf446885e55b45ef91f01e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:31:15.813235 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c76d264-7bb2-4ded-8a2b-10debc6cf39b service cinder] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:31:15.813523 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c76d264-7bb2-4ded-8a2b-10debc6cf39b service cinder] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:31:15.813803 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c76d264-7bb2-4ded-8a2b-10debc6cf39b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:31:15.815479 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-9c76d264-7bb2-4ded-8a2b-10debc6cf39b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:31:15.921990 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c76d264-7bb2-4ded-8a2b-10debc6cf39b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:31:15.923490 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c76d264-7bb2-4ded-8a2b-10debc6cf39b service cinder] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:31:15.927617 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:31:15.943297 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 759/1505] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:31:15 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:31:15.987194 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d15a97-18fe-4652-ba8b-ca29c0187916 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:31:15.987635 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d15a97-18fe-4652-ba8b-ca29c0187916 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:31:15.987740 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d15a97-18fe-4652-ba8b-ca29c0187916 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:31:15.989870 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 747/1506] 10.222.0.66 () {52 vars in 829 bytes} [Mon Jul 18 02:31:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:31:15.993504 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c76d264-7bb2-4ded-8a2b-10debc6cf39b service cinder] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:31:15.994730 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-9c76d264-7bb2-4ded-8a2b-10debc6cf39b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:31:16.092926 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c76d264-7bb2-4ded-8a2b-10debc6cf39b service cinder] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:31:16.187015 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c9f32e-e88e-4ea8-9dcc-946de7a9e341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:31:16.187792 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c9f32e-e88e-4ea8-9dcc-946de7a9e341 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:31:16.188075 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c9f32e-e88e-4ea8-9dcc-946de7a9e341 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:31:16.188349 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c9f32e-e88e-4ea8-9dcc-946de7a9e341 admin admin] PATH_INFO: `/v3/users/f140d30f190f447580ea80d59a082241` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:31:16.201037 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c9f32e-e88e-4ea8-9dcc-946de7a9e341 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f140d30f190f447580ea80d59a082241)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:31:16.202198 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: WARNING py.warnings [None req-76c9f32e-e88e-4ea8-9dcc-946de7a9e341 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 02:31:16.202198 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: warnings.warn(msg) Jul 18 02:31:16.202198 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: Jul 18 02:31:16.203422 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c9f32e-e88e-4ea8-9dcc-946de7a9e341 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:31:16.212959 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:31:16.407976 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-76c9f32e-e88e-4ea8-9dcc-946de7a9e341 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f140d30f190f447580ea80d59a082241'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:31:16.464685 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-76c9f32e-e88e-4ea8-9dcc-946de7a9e341 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f140d30f190f447580ea80d59a082241'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:31:16.478328 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-76c9f32e-e88e-4ea8-9dcc-946de7a9e341 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f140d30f190f447580ea80d59a082241'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:31:16.478769 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-76c9f32e-e88e-4ea8-9dcc-946de7a9e341 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f140d30f190f447580ea80d59a082241'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:31:16.497463 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-76c9f32e-e88e-4ea8-9dcc-946de7a9e341 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f140d30f190f447580ea80d59a082241'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:31:16.503444 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 760/1507] 10.222.0.66 () {58 vars in 1223 bytes} [Mon Jul 18 02:31:15 2022] DELETE /identity/v3/users/f140d30f190f447580ea80d59a082241 => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:31:16.508433 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0865b57-be83-432e-a203-86221fb858b8 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:31:16.508714 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0865b57-be83-432e-a203-86221fb858b8 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:31:16.508990 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0865b57-be83-432e-a203-86221fb858b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:31:16.564347 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-f0865b57-be83-432e-a203-86221fb858b8 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:31:16.565540 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-f0865b57-be83-432e-a203-86221fb858b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:31:16.672971 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:31:16.685703 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 748/1508] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:31:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:31:16.695829 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c9f32e-e88e-4ea8-9dcc-946de7a9e341 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:31:16.697038 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-76c9f32e-e88e-4ea8-9dcc-946de7a9e341 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:31:16.796225 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c9f32e-e88e-4ea8-9dcc-946de7a9e341 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:31:16.850736 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:31:16.869263 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c76ecd-97b9-4898-af9c-08101081bc41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eaf19a259c2741cea375a38763e33cdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:31:16.869903 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c76ecd-97b9-4898-af9c-08101081bc41 service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:31:16.870216 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c76ecd-97b9-4898-af9c-08101081bc41 service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:31:16.870490 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c76ecd-97b9-4898-af9c-08101081bc41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:31:16.872145 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-21c76ecd-97b9-4898-af9c-08101081bc41 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:31:16.980057 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c76ecd-97b9-4898-af9c-08101081bc41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:31:16.981502 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c76ecd-97b9-4898-af9c-08101081bc41 service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:31:17.002188 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 761/1509] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:31:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:31:17.455258 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0865b57-be83-432e-a203-86221fb858b8 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:31:17.456513 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-f0865b57-be83-432e-a203-86221fb858b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:31:17.468935 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0865b57-be83-432e-a203-86221fb858b8 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:31:17.566936 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a114ca9d-efa3-422c-b2ac-e0cb8077308e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:31:17.567798 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a114ca9d-efa3-422c-b2ac-e0cb8077308e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:31:17.568080 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a114ca9d-efa3-422c-b2ac-e0cb8077308e admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:31:17.568355 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a114ca9d-efa3-422c-b2ac-e0cb8077308e admin admin] PATH_INFO: `/v3/projects/4468af83d6fb446fa69cd11024d8c564` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:31:17.579664 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a114ca9d-efa3-422c-b2ac-e0cb8077308e admin admin] RBAC: Authorizing `identity:delete_project(project_id=4468af83d6fb446fa69cd11024d8c564)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:31:17.580685 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: WARNING py.warnings [None req-a114ca9d-efa3-422c-b2ac-e0cb8077308e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 02:31:17.580685 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: warnings.warn(msg) Jul 18 02:31:17.580685 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: Jul 18 02:31:17.584813 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a114ca9d-efa3-422c-b2ac-e0cb8077308e admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:31:17.691599 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-a114ca9d-efa3-422c-b2ac-e0cb8077308e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4468af83d6fb446fa69cd11024d8c564', 'request_id': 'req-a114ca9d-efa3-422c-b2ac-e0cb8077308e'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:31:17.697685 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-a114ca9d-efa3-422c-b2ac-e0cb8077308e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4468af83d6fb446fa69cd11024d8c564', 'request_id': 'req-a114ca9d-efa3-422c-b2ac-e0cb8077308e'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:31:17.714075 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-a114ca9d-efa3-422c-b2ac-e0cb8077308e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4468af83d6fb446fa69cd11024d8c564', 'request_id': 'req-a114ca9d-efa3-422c-b2ac-e0cb8077308e'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:31:17.807652 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-a114ca9d-efa3-422c-b2ac-e0cb8077308e admin admin] The token cache is being invalidate because project 4468af83d6fb446fa69cd11024d8c564 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 02:31:17.807945 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-a114ca9d-efa3-422c-b2ac-e0cb8077308e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:31:17.808373 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-a114ca9d-efa3-422c-b2ac-e0cb8077308e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:31:17.809842 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 749/1510] 10.222.0.66 () {58 vars in 1232 bytes} [Mon Jul 18 02:31:17 2022] DELETE /identity/v3/projects/4468af83d6fb446fa69cd11024d8c564 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:31:26.999894 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=62 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:31:27.805614 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:32:58.455567 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084fb458-8085-4158-a6e4-e9f65aad48f0 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:32:58.456172 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084fb458-8085-4158-a6e4-e9f65aad48f0 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:32:58.456172 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084fb458-8085-4158-a6e4-e9f65aad48f0 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:32:58.457500 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 762/1511] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 02:32:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:32:58.462316 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7182bbdc-3802-410c-893e-2ba11397eac8 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:32:58.463351 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7182bbdc-3802-410c-893e-2ba11397eac8 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:32:58.463609 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7182bbdc-3802-410c-893e-2ba11397eac8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:32:58.542861 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-7182bbdc-3802-410c-893e-2ba11397eac8 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:32:58.544071 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-7182bbdc-3802-410c-893e-2ba11397eac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:32:58.647906 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 750/1512] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 02:32:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:32:58.803603 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084fb458-8085-4158-a6e4-e9f65aad48f0 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:32:58.805679 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-084fb458-8085-4158-a6e4-e9f65aad48f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:32:58.927131 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084fb458-8085-4158-a6e4-e9f65aad48f0 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:32:58.933688 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2381f6a1-e1f9-44bf-a9a6-1b2a52938888 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80b138831bc54c7b986d50092d62d4af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:32:58.934360 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2381f6a1-e1f9-44bf-a9a6-1b2a52938888 service nova] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:32:58.934645 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2381f6a1-e1f9-44bf-a9a6-1b2a52938888 service nova] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:32:58.934919 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2381f6a1-e1f9-44bf-a9a6-1b2a52938888 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:32:58.943045 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2381f6a1-e1f9-44bf-a9a6-1b2a52938888 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:32:58.944622 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2381f6a1-e1f9-44bf-a9a6-1b2a52938888 service nova] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:32:58.954434 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 763/1513] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:32:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:32:59.784895 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775f55d6-2dad-43d4-bca6-2ffe19ebb2ad None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:32:59.785405 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775f55d6-2dad-43d4-bca6-2ffe19ebb2ad None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:32:59.785520 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775f55d6-2dad-43d4-bca6-2ffe19ebb2ad None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:32:59.786885 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 751/1514] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 02:32:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:32:59.792136 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85569e4-f4c1-403d-8182-62f1a16da2d3 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:32:59.792579 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85569e4-f4c1-403d-8182-62f1a16da2d3 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:32:59.792691 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85569e4-f4c1-403d-8182-62f1a16da2d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:32:59.811488 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-c85569e4-f4c1-403d-8182-62f1a16da2d3 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:32:59.812701 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-c85569e4-f4c1-403d-8182-62f1a16da2d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:32:59.820594 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 764/1515] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 02:32:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:32:59.972560 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775f55d6-2dad-43d4-bca6-2ffe19ebb2ad None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:32:59.987433 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775f55d6-2dad-43d4-bca6-2ffe19ebb2ad None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:32:59.992810 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c2ec34-8b47-4675-9306-b211a47a7a57 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80b138831bc54c7b986d50092d62d4af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:32:59.993442 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c2ec34-8b47-4675-9306-b211a47a7a57 service nova] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:32:59.993730 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c2ec34-8b47-4675-9306-b211a47a7a57 service nova] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:32:59.994117 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c2ec34-8b47-4675-9306-b211a47a7a57 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:33:00.002299 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c2ec34-8b47-4675-9306-b211a47a7a57 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:33:00.003898 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c2ec34-8b47-4675-9306-b211a47a7a57 service nova] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:33:00.013295 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 752/1516] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:32:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:33:00.557042 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79a8236-3417-4b11-bed0-e456f60ed4ec None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:33:00.557337 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79a8236-3417-4b11-bed0-e456f60ed4ec None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:33:00.557622 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79a8236-3417-4b11-bed0-e456f60ed4ec None None] PATH_INFO: `/v3` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:33:00.558994 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 765/1517] 10.222.0.72 () {54 vars in 925 bytes} [Mon Jul 18 02:33:00 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:33:00.565457 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc315bf-9dc0-4fa1-8ed1-91e0afd257be None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:33:00.565745 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc315bf-9dc0-4fa1-8ed1-91e0afd257be None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:33:00.566025 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc315bf-9dc0-4fa1-8ed1-91e0afd257be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:33:00.623599 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-1bc315bf-9dc0-4fa1-8ed1-91e0afd257be None None] MFA Rules not processed for user `88936b31709b429f8b6777c673ed7b8a`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:33:00.624806 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-1bc315bf-9dc0-4fa1-8ed1-91e0afd257be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:33:00.642307 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:33:00.699863 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 753/1518] 10.222.0.72 () {58 vars in 1015 bytes} [Mon Jul 18 02:33:00 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:33:09.806289 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=56 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:33:10.153937 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c44e0e8-9e09-47be-8d2f-96b0cecc0fe1 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:33:10.154261 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c44e0e8-9e09-47be-8d2f-96b0cecc0fe1 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:33:10.154544 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c44e0e8-9e09-47be-8d2f-96b0cecc0fe1 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:33:10.155935 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 766/1519] 10.222.0.66 () {54 vars in 961 bytes} [Mon Jul 18 02:33:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:33:10.160934 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2422a650-8639-45c0-87cd-7945243e8e3c None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:33:10.161401 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2422a650-8639-45c0-87cd-7945243e8e3c None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:33:10.161505 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2422a650-8639-45c0-87cd-7945243e8e3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:33:10.179956 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-2422a650-8639-45c0-87cd-7945243e8e3c None None] MFA Rules not processed for user `88936b31709b429f8b6777c673ed7b8a`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:33:10.181926 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-2422a650-8639-45c0-87cd-7945243e8e3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:33:10.190791 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 754/1520] 10.222.0.66 () {58 vars in 1058 bytes} [Mon Jul 18 02:33:10 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:33:10.198393 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c44e0e8-9e09-47be-8d2f-96b0cecc0fe1 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:33:10.212071 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c44e0e8-9e09-47be-8d2f-96b0cecc0fe1 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:33:10.217318 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91b8401-6386-4f20-924a-ed77e60c326d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88936b31709b429f8b6777c673ed7b8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:33:10.217957 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91b8401-6386-4f20-924a-ed77e60c326d service placement] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:33:10.218283 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91b8401-6386-4f20-924a-ed77e60c326d service placement] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:33:10.218562 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91b8401-6386-4f20-924a-ed77e60c326d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:33:10.227493 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91b8401-6386-4f20-924a-ed77e60c326d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:33:10.229057 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91b8401-6386-4f20-924a-ed77e60c326d service placement] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:33:10.238578 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 767/1521] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:33:10 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:33:11.256541 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199d7021-c4a8-432c-b41c-10d3bc3660ec None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:33:11.257108 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199d7021-c4a8-432c-b41c-10d3bc3660ec None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:33:11.257108 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199d7021-c4a8-432c-b41c-10d3bc3660ec None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:33:11.259407 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 755/1522] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 02:33:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:33:11.264624 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38414ebf-f12d-4d84-bb10-0f43c3b4bca0 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:33:11.265082 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38414ebf-f12d-4d84-bb10-0f43c3b4bca0 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:33:11.265195 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38414ebf-f12d-4d84-bb10-0f43c3b4bca0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:33:11.284102 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-38414ebf-f12d-4d84-bb10-0f43c3b4bca0 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:33:11.285297 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-38414ebf-f12d-4d84-bb10-0f43c3b4bca0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:33:11.293358 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 768/1523] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 02:33:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:33:11.449761 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199d7021-c4a8-432c-b41c-10d3bc3660ec None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:33:11.464361 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199d7021-c4a8-432c-b41c-10d3bc3660ec None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:33:11.469762 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfee3c0-6737-42b9-ab14-770d8727c4d2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80b138831bc54c7b986d50092d62d4af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:33:11.470438 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfee3c0-6737-42b9-ab14-770d8727c4d2 service nova] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:33:11.470718 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfee3c0-6737-42b9-ab14-770d8727c4d2 service nova] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:33:11.470989 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfee3c0-6737-42b9-ab14-770d8727c4d2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:33:11.479235 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfee3c0-6737-42b9-ab14-770d8727c4d2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:33:11.480815 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfee3c0-6737-42b9-ab14-770d8727c4d2 service nova] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:33:11.490105 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 756/1524] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:33:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:33:12.312203 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb30843f-1793-42db-a707-cf827f2ef25b None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:33:12.312807 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb30843f-1793-42db-a707-cf827f2ef25b None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:33:12.312807 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb30843f-1793-42db-a707-cf827f2ef25b None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:33:12.314272 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 769/1525] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 02:33:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:33:12.319062 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717f0636-f5d0-4a0b-8545-535388a0b77a None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:33:12.319517 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717f0636-f5d0-4a0b-8545-535388a0b77a None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:33:12.319684 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717f0636-f5d0-4a0b-8545-535388a0b77a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:33:12.337317 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-717f0636-f5d0-4a0b-8545-535388a0b77a None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:33:12.338521 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-717f0636-f5d0-4a0b-8545-535388a0b77a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:33:12.346539 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 757/1526] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 02:33:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:33:12.496916 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb30843f-1793-42db-a707-cf827f2ef25b None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:33:12.511327 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb30843f-1793-42db-a707-cf827f2ef25b None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:33:12.516717 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6761c4-e869-4b3d-a96f-9a95410aa866 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80b138831bc54c7b986d50092d62d4af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:33:12.517393 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6761c4-e869-4b3d-a96f-9a95410aa866 service nova] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:33:12.517676 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6761c4-e869-4b3d-a96f-9a95410aa866 service nova] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:33:12.517952 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6761c4-e869-4b3d-a96f-9a95410aa866 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:33:12.526459 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6761c4-e869-4b3d-a96f-9a95410aa866 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:33:12.528299 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6761c4-e869-4b3d-a96f-9a95410aa866 service nova] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:33:12.537772 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 770/1527] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:33:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:33:16.557856 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a6d09d-2d9c-4778-807e-f5938739f725 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:33:16.558466 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a6d09d-2d9c-4778-807e-f5938739f725 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:33:16.558466 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a6d09d-2d9c-4778-807e-f5938739f725 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:33:16.559743 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 758/1528] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 02:33:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:33:16.565091 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f4fd2c-a6ad-43f3-8468-5841c20a2d33 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:33:16.565563 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f4fd2c-a6ad-43f3-8468-5841c20a2d33 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:33:16.565672 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f4fd2c-a6ad-43f3-8468-5841c20a2d33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:33:16.583799 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-75f4fd2c-a6ad-43f3-8468-5841c20a2d33 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:33:16.585008 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-75f4fd2c-a6ad-43f3-8468-5841c20a2d33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:33:16.592910 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 771/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 02:33:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:33:16.630792 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a6d09d-2d9c-4778-807e-f5938739f725 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:33:16.632478 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-47a6d09d-2d9c-4778-807e-f5938739f725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:33:16.711242 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a6d09d-2d9c-4778-807e-f5938739f725 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:33:16.714795 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01390972-33be-4597-ad30-a706cbbe6cf8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eaf19a259c2741cea375a38763e33cdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:33:16.715434 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01390972-33be-4597-ad30-a706cbbe6cf8 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:33:16.715716 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01390972-33be-4597-ad30-a706cbbe6cf8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:33:16.715992 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01390972-33be-4597-ad30-a706cbbe6cf8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:33:16.725017 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01390972-33be-4597-ad30-a706cbbe6cf8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:33:16.726514 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01390972-33be-4597-ad30-a706cbbe6cf8 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:33:16.735798 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 759/1530] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:33:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:33:18.740337 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e547d263-7034-44c4-a818-90fc8e5adee7 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:33:18.740947 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e547d263-7034-44c4-a818-90fc8e5adee7 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:33:18.741175 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e547d263-7034-44c4-a818-90fc8e5adee7 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:33:18.743177 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 772/1531] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 02:33:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:33:18.747985 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b315da-311c-4735-8e1a-95b7ab4bbd6a None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:33:18.748450 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b315da-311c-4735-8e1a-95b7ab4bbd6a None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:33:18.748562 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b315da-311c-4735-8e1a-95b7ab4bbd6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:33:18.767417 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-c8b315da-311c-4735-8e1a-95b7ab4bbd6a None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:33:18.768642 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-c8b315da-311c-4735-8e1a-95b7ab4bbd6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:33:18.777898 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 760/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 02:33:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:33:18.814652 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e547d263-7034-44c4-a818-90fc8e5adee7 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:33:18.816356 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-e547d263-7034-44c4-a818-90fc8e5adee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:33:18.829117 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e547d263-7034-44c4-a818-90fc8e5adee7 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:33:18.832637 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646246fc-41ac-454f-8650-d6c261fba788 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eaf19a259c2741cea375a38763e33cdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:33:18.833273 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646246fc-41ac-454f-8650-d6c261fba788 service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:33:18.833573 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646246fc-41ac-454f-8650-d6c261fba788 service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:33:18.833829 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646246fc-41ac-454f-8650-d6c261fba788 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:33:18.843668 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646246fc-41ac-454f-8650-d6c261fba788 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:33:18.845148 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646246fc-41ac-454f-8650-d6c261fba788 service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:33:18.855484 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 773/1533] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:33:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:33:22.339939 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959a165b-f15b-454a-a1c1-48576ecbb56c None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:33:22.340515 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959a165b-f15b-454a-a1c1-48576ecbb56c None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:33:22.340515 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959a165b-f15b-454a-a1c1-48576ecbb56c None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:33:22.341894 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 761/1534] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 02:33:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:33:22.347148 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ee4a98-d039-4a07-8965-fe842830dfae None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:33:22.347535 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ee4a98-d039-4a07-8965-fe842830dfae None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:33:22.347711 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ee4a98-d039-4a07-8965-fe842830dfae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:33:22.366294 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-e2ee4a98-d039-4a07-8965-fe842830dfae None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:33:22.368071 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-e2ee4a98-d039-4a07-8965-fe842830dfae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:33:22.375810 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 774/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 02:33:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:33:22.680152 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959a165b-f15b-454a-a1c1-48576ecbb56c None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:33:22.681848 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-959a165b-f15b-454a-a1c1-48576ecbb56c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:33:22.747795 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:33:22.761484 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959a165b-f15b-454a-a1c1-48576ecbb56c None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:33:22.767556 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9fc8c9-b97a-4fa4-a48c-2c9bbcc9f4ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd49388facdf446885e55b45ef91f01e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:33:22.767556 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9fc8c9-b97a-4fa4-a48c-2c9bbcc9f4ef service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:33:22.767556 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9fc8c9-b97a-4fa4-a48c-2c9bbcc9f4ef service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:33:22.767556 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9fc8c9-b97a-4fa4-a48c-2c9bbcc9f4ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:33:22.775419 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9fc8c9-b97a-4fa4-a48c-2c9bbcc9f4ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:33:22.776860 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9fc8c9-b97a-4fa4-a48c-2c9bbcc9f4ef service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:33:22.787174 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 762/1536] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:33:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:33:24.839114 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e551f2-9b0c-495c-98fa-d82da5b8d2fa None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:33:24.839682 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e551f2-9b0c-495c-98fa-d82da5b8d2fa None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:33:24.839682 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e551f2-9b0c-495c-98fa-d82da5b8d2fa None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:33:24.841052 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 775/1537] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 02:33:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:33:24.846367 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f939d0c0-b8e8-4fc4-a8e2-e247f4b54b66 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:33:24.846828 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f939d0c0-b8e8-4fc4-a8e2-e247f4b54b66 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:33:24.847001 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f939d0c0-b8e8-4fc4-a8e2-e247f4b54b66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:33:24.873571 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-f939d0c0-b8e8-4fc4-a8e2-e247f4b54b66 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:33:24.873571 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-f939d0c0-b8e8-4fc4-a8e2-e247f4b54b66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:33:24.874749 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 763/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 02:33:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:33:25.185457 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e551f2-9b0c-495c-98fa-d82da5b8d2fa None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:33:25.187215 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-a7e551f2-9b0c-495c-98fa-d82da5b8d2fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:33:25.200358 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e551f2-9b0c-495c-98fa-d82da5b8d2fa None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:33:25.203929 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8db726e-7761-4a40-a17b-caaa797aca83 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd49388facdf446885e55b45ef91f01e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:33:25.204566 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8db726e-7761-4a40-a17b-caaa797aca83 service cinder] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:33:25.204850 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8db726e-7761-4a40-a17b-caaa797aca83 service cinder] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:33:25.205129 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8db726e-7761-4a40-a17b-caaa797aca83 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:33:25.214297 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8db726e-7761-4a40-a17b-caaa797aca83 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:33:25.215728 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8db726e-7761-4a40-a17b-caaa797aca83 service cinder] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:33:25.225042 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 776/1539] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:33:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:33:34.859747 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=11 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:33:35.220892 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=33 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:34:53.836258 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2de0bc-8569-472d-8a7e-be98fad7137a None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:34:53.837605 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2de0bc-8569-472d-8a7e-be98fad7137a None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:34:53.837605 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2de0bc-8569-472d-8a7e-be98fad7137a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:34:53.877216 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-3c2de0bc-8569-472d-8a7e-be98fad7137a None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:34:53.878528 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-3c2de0bc-8569-472d-8a7e-be98fad7137a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:34:53.892435 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 764/1540] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:34:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 02:34:53.902008 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f7100b-f003-4e21-b861-29e30f2049de None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:34:53.902640 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f7100b-f003-4e21-b861-29e30f2049de None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:34:53.902891 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f7100b-f003-4e21-b861-29e30f2049de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:34:53.936637 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-96f7100b-f003-4e21-b861-29e30f2049de None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:34:53.939360 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-96f7100b-f003-4e21-b861-29e30f2049de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:34:53.952669 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 777/1541] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:34:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 02:34:53.957727 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2de0bc-8569-472d-8a7e-be98fad7137a None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:34:53.972255 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2de0bc-8569-472d-8a7e-be98fad7137a None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:34:53.983033 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244036f4-29e7-40a4-adc5-47a953cb38a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:34:53.983784 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244036f4-29e7-40a4-adc5-47a953cb38a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:34:53.984075 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244036f4-29e7-40a4-adc5-47a953cb38a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:34:53.984352 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244036f4-29e7-40a4-adc5-47a953cb38a1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:34:53.985653 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244036f4-29e7-40a4-adc5-47a953cb38a1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:34:53.985967 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244036f4-29e7-40a4-adc5-47a953cb38a1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:34:53.987765 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244036f4-29e7-40a4-adc5-47a953cb38a1 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:34:54.018974 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 765/1542] 10.222.0.66 () {58 vars in 1152 bytes} [Mon Jul 18 02:34:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:34:54.028293 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92201b7-a28f-4330-8448-3ba1de835cf8 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:34:54.028626 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92201b7-a28f-4330-8448-3ba1de835cf8 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:34:54.028861 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92201b7-a28f-4330-8448-3ba1de835cf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:34:54.035607 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244036f4-29e7-40a4-adc5-47a953cb38a1 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:34:54.052319 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244036f4-29e7-40a4-adc5-47a953cb38a1 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:34:54.059648 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-d92201b7-a28f-4330-8448-3ba1de835cf8 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:34:54.060840 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-d92201b7-a28f-4330-8448-3ba1de835cf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:34:54.067511 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3c268b-0ba6-4f2d-9d79-0636cfa86458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:34:54.068191 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3c268b-0ba6-4f2d-9d79-0636cfa86458 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:34:54.068477 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3c268b-0ba6-4f2d-9d79-0636cfa86458 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:34:54.068754 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3c268b-0ba6-4f2d-9d79-0636cfa86458 admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:34:54.069454 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3c268b-0ba6-4f2d-9d79-0636cfa86458 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:34:54.070509 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: WARNING py.warnings [None req-bb3c268b-0ba6-4f2d-9d79-0636cfa86458 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 02:34:54.070509 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: warnings.warn(msg) Jul 18 02:34:54.070509 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: Jul 18 02:34:54.071910 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3c268b-0ba6-4f2d-9d79-0636cfa86458 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:34:54.077092 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 778/1543] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:34:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 02:34:54.089364 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186022e2-3b3d-453e-9180-54744f512edf None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:34:54.089714 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186022e2-3b3d-453e-9180-54744f512edf None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:34:54.089929 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186022e2-3b3d-453e-9180-54744f512edf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:34:54.118461 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-186022e2-3b3d-453e-9180-54744f512edf None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:34:54.119723 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-186022e2-3b3d-453e-9180-54744f512edf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:34:54.143656 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 779/1544] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:34:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 02:34:54.143656 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd680448-4f89-4d77-be17-ee4b86acae4b None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:34:54.143656 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd680448-4f89-4d77-be17-ee4b86acae4b None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:34:54.143656 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd680448-4f89-4d77-be17-ee4b86acae4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:34:54.166363 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-cd680448-4f89-4d77-be17-ee4b86acae4b None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:34:54.167588 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-cd680448-4f89-4d77-be17-ee4b86acae4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:34:54.182740 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 780/1545] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:34:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:34:54.183602 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd680448-4f89-4d77-be17-ee4b86acae4b None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:34:54.197079 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd680448-4f89-4d77-be17-ee4b86acae4b None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:34:54.210108 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796935fa-6fb9-4563-9b48-c6227d24f5ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:34:54.210713 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 766/1546] 10.222.0.66 () {60 vars in 1152 bytes} [Mon Jul 18 02:34:54 2022] POST /identity/v3/projects => generated 357 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:34:54.210843 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796935fa-6fb9-4563-9b48-c6227d24f5ad admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:34:54.211066 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796935fa-6fb9-4563-9b48-c6227d24f5ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:34:54.211347 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796935fa-6fb9-4563-9b48-c6227d24f5ad admin admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:34:54.212050 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d974e55-0705-4be0-8948-bcb5381c490a None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:34:54.212253 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796935fa-6fb9-4563-9b48-c6227d24f5ad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:34:54.212369 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d974e55-0705-4be0-8948-bcb5381c490a None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:34:54.212572 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796935fa-6fb9-4563-9b48-c6227d24f5ad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:34:54.212689 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d974e55-0705-4be0-8948-bcb5381c490a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:34:54.213602 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: WARNING py.warnings [None req-796935fa-6fb9-4563-9b48-c6227d24f5ad admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 18 02:34:54.213602 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: warnings.warn(msg) Jul 18 02:34:54.213602 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: Jul 18 02:34:54.214637 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796935fa-6fb9-4563-9b48-c6227d24f5ad admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:34:54.230770 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 781/1547] 10.222.0.66 () {58 vars in 1152 bytes} [Mon Jul 18 02:34:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:34:54.232101 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914addb1-b6bb-4277-88f6-0e29eddf1cd1 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:34:54.232580 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914addb1-b6bb-4277-88f6-0e29eddf1cd1 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:34:54.232871 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914addb1-b6bb-4277-88f6-0e29eddf1cd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:34:54.253297 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-5d974e55-0705-4be0-8948-bcb5381c490a None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:34:54.254679 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-5d974e55-0705-4be0-8948-bcb5381c490a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:34:54.260891 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-914addb1-b6bb-4277-88f6-0e29eddf1cd1 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:34:54.262515 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-914addb1-b6bb-4277-88f6-0e29eddf1cd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:34:54.332557 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 782/1548] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:34:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:34:54.333699 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914addb1-b6bb-4277-88f6-0e29eddf1cd1 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:34:54.344908 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914addb1-b6bb-4277-88f6-0e29eddf1cd1 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:34:54.345537 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 767/1549] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:34:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:34:54.347042 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd2679f-d64e-4162-8ab6-75a65bc417a5 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:34:54.347432 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd2679f-d64e-4162-8ab6-75a65bc417a5 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:34:54.347809 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd2679f-d64e-4162-8ab6-75a65bc417a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:34:54.355616 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce56672d-f474-4701-8a51-14687e56cedf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:34:54.356915 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce56672d-f474-4701-8a51-14687e56cedf admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:34:54.358419 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce56672d-f474-4701-8a51-14687e56cedf admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:34:54.358875 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce56672d-f474-4701-8a51-14687e56cedf admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:34:54.361306 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce56672d-f474-4701-8a51-14687e56cedf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:34:54.363266 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: WARNING py.warnings [None req-ce56672d-f474-4701-8a51-14687e56cedf admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 02:34:54.363266 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: warnings.warn(msg) Jul 18 02:34:54.363266 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: Jul 18 02:34:54.364935 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce56672d-f474-4701-8a51-14687e56cedf admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:34:54.381737 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-dcd2679f-d64e-4162-8ab6-75a65bc417a5 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:34:54.383094 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-dcd2679f-d64e-4162-8ab6-75a65bc417a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:34:54.392026 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 768/1550] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:34:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:34:54.392806 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd2679f-d64e-4162-8ab6-75a65bc417a5 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:34:54.403202 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd2679f-d64e-4162-8ab6-75a65bc417a5 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:34:54.408033 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f19537-3ed5-49b0-8020-dbd09ff30aaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:34:54.408681 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f19537-3ed5-49b0-8020-dbd09ff30aaf admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:34:54.408962 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f19537-3ed5-49b0-8020-dbd09ff30aaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:34:54.409234 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f19537-3ed5-49b0-8020-dbd09ff30aaf admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:34:54.410050 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f19537-3ed5-49b0-8020-dbd09ff30aaf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:34:54.412015 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f19537-3ed5-49b0-8020-dbd09ff30aaf admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:34:54.462094 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 783/1551] 10.222.0.66 () {60 vars in 1143 bytes} [Mon Jul 18 02:34:54 2022] POST /identity/v3/users => generated 279 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:34:54.463680 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1fb18f-d17b-46d8-8204-92707ecf4f40 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:34:54.464055 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1fb18f-d17b-46d8-8204-92707ecf4f40 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:34:54.465167 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1fb18f-d17b-46d8-8204-92707ecf4f40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:34:54.484553 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-8d1fb18f-d17b-46d8-8204-92707ecf4f40 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:34:54.485831 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-8d1fb18f-d17b-46d8-8204-92707ecf4f40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:34:54.493848 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 784/1552] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:34:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:34:54.494668 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1fb18f-d17b-46d8-8204-92707ecf4f40 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:34:54.505041 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1fb18f-d17b-46d8-8204-92707ecf4f40 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:34:54.509729 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03239eb0-c177-4655-afe1-89b997d2ebf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:34:54.511912 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03239eb0-c177-4655-afe1-89b997d2ebf2 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:34:54.511912 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03239eb0-c177-4655-afe1-89b997d2ebf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:34:54.511912 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03239eb0-c177-4655-afe1-89b997d2ebf2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:34:54.511912 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03239eb0-c177-4655-afe1-89b997d2ebf2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:34:54.511912 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03239eb0-c177-4655-afe1-89b997d2ebf2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:34:54.513502 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03239eb0-c177-4655-afe1-89b997d2ebf2 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:34:54.526264 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 785/1553] 10.222.0.66 () {58 vars in 1152 bytes} [Mon Jul 18 02:34:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:34:54.527641 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9624a23b-bde0-4007-9124-1f8307e2d773 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:34:54.527881 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9624a23b-bde0-4007-9124-1f8307e2d773 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:34:54.528156 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9624a23b-bde0-4007-9124-1f8307e2d773 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:34:54.534525 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 769/1554] 10.222.0.66 () {60 vars in 1152 bytes} [Mon Jul 18 02:34:54 2022] POST /identity/v3/projects => generated 377 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:34:54.534525 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa41d5b3-689b-42fe-a0cb-e43d3ae46654 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:34:54.534525 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa41d5b3-689b-42fe-a0cb-e43d3ae46654 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:34:54.534525 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa41d5b3-689b-42fe-a0cb-e43d3ae46654 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:34:54.549657 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-9624a23b-bde0-4007-9124-1f8307e2d773 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:34:54.550882 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-9624a23b-bde0-4007-9124-1f8307e2d773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:34:54.551449 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-aa41d5b3-689b-42fe-a0cb-e43d3ae46654 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:34:54.552610 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-aa41d5b3-689b-42fe-a0cb-e43d3ae46654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:34:54.619433 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 786/1555] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:34:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:34:54.621681 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d96501-218d-4f8d-8e6e-86fa30e7f5dc None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:34:54.621915 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 770/1556] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:34:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:34:54.622197 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d96501-218d-4f8d-8e6e-86fa30e7f5dc None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:34:54.622652 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d96501-218d-4f8d-8e6e-86fa30e7f5dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:34:54.624596 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a633a2a-ab11-4d91-bf63-5d00a526118d None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:34:54.624746 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a633a2a-ab11-4d91-bf63-5d00a526118d None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:34:54.625216 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a633a2a-ab11-4d91-bf63-5d00a526118d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:34:54.643594 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-03d96501-218d-4f8d-8e6e-86fa30e7f5dc None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:34:54.644809 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-03d96501-218d-4f8d-8e6e-86fa30e7f5dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:34:54.645282 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-9a633a2a-ab11-4d91-bf63-5d00a526118d None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:34:54.646485 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-9a633a2a-ab11-4d91-bf63-5d00a526118d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:34:54.653200 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 787/1557] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:34:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:34:54.654025 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d96501-218d-4f8d-8e6e-86fa30e7f5dc None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:34:54.659656 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 771/1558] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:34:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:34:54.661135 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9157f76c-84c8-411e-a8a7-7869446437ba None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:34:54.661551 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9157f76c-84c8-411e-a8a7-7869446437ba None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:34:54.661836 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9157f76c-84c8-411e-a8a7-7869446437ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:34:54.667691 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d96501-218d-4f8d-8e6e-86fa30e7f5dc None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:34:54.673515 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5081d78-dcb4-4946-8c92-6b6df9e8ec39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:34:54.674217 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5081d78-dcb4-4946-8c92-6b6df9e8ec39 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:34:54.674512 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5081d78-dcb4-4946-8c92-6b6df9e8ec39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:34:54.674791 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5081d78-dcb4-4946-8c92-6b6df9e8ec39 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:34:54.675522 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5081d78-dcb4-4946-8c92-6b6df9e8ec39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:34:54.677148 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5081d78-dcb4-4946-8c92-6b6df9e8ec39 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:34:54.680737 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-9157f76c-84c8-411e-a8a7-7869446437ba None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:34:54.681919 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-9157f76c-84c8-411e-a8a7-7869446437ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:34:54.688463 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 788/1559] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:34:54 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:34:54.689139 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5081d78-dcb4-4946-8c92-6b6df9e8ec39 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:34:54.690754 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 772/1560] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:34:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:34:54.691512 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9157f76c-84c8-411e-a8a7-7869446437ba None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:34:54.701042 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9157f76c-84c8-411e-a8a7-7869446437ba None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:34:54.701210 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5081d78-dcb4-4946-8c92-6b6df9e8ec39 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:34:54.706048 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2442a0-ef65-4395-af0f-8f3fb44c9bd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:34:54.706537 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c285c6da-967c-4f01-9813-9ab9bbc98356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:34:54.706733 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2442a0-ef65-4395-af0f-8f3fb44c9bd9 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:34:54.707018 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2442a0-ef65-4395-af0f-8f3fb44c9bd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:34:54.707195 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c285c6da-967c-4f01-9813-9ab9bbc98356 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:34:54.707302 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2442a0-ef65-4395-af0f-8f3fb44c9bd9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:34:54.707483 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c285c6da-967c-4f01-9813-9ab9bbc98356 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:34:54.707762 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c285c6da-967c-4f01-9813-9ab9bbc98356 admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:34:54.707967 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2442a0-ef65-4395-af0f-8f3fb44c9bd9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:34:54.708448 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c285c6da-967c-4f01-9813-9ab9bbc98356 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:34:54.709879 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2442a0-ef65-4395-af0f-8f3fb44c9bd9 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:34:54.710398 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c285c6da-967c-4f01-9813-9ab9bbc98356 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:34:54.713192 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:34:54.718653 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:34:54.811817 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 789/1561] 10.222.0.66 () {60 vars in 1152 bytes} [Mon Jul 18 02:34:54 2022] POST /identity/v3/projects => generated 367 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:34:54.813406 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2442a0-ef65-4395-af0f-8f3fb44c9bd9 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:34:54.921414 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2442a0-ef65-4395-af0f-8f3fb44c9bd9 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:34:54.926754 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a7ee87-2a89-4045-a767-f9825006c820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:34:54.927413 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a7ee87-2a89-4045-a767-f9825006c820 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:34:54.927738 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a7ee87-2a89-4045-a767-f9825006c820 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:34:54.928092 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a7ee87-2a89-4045-a767-f9825006c820 admin admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:34:54.928819 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a7ee87-2a89-4045-a767-f9825006c820 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:34:54.929129 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a7ee87-2a89-4045-a767-f9825006c820 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:34:54.930793 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a7ee87-2a89-4045-a767-f9825006c820 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:34:54.942988 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 790/1562] 10.222.0.66 () {58 vars in 1152 bytes} [Mon Jul 18 02:34:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:34:54.944183 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fdd9a6-3202-4123-b096-c58af3d33c26 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:34:54.944475 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fdd9a6-3202-4123-b096-c58af3d33c26 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:34:54.944756 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fdd9a6-3202-4123-b096-c58af3d33c26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:34:54.961916 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-98fdd9a6-3202-4123-b096-c58af3d33c26 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:34:54.963138 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-98fdd9a6-3202-4123-b096-c58af3d33c26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:34:54.971785 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 791/1563] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:34:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:34:54.972989 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ad921f-677f-455f-a24c-e64eef9cdb8c None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:34:54.973279 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ad921f-677f-455f-a24c-e64eef9cdb8c None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:34:54.973651 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ad921f-677f-455f-a24c-e64eef9cdb8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:34:54.992571 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-88ad921f-677f-455f-a24c-e64eef9cdb8c None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:34:54.993997 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-88ad921f-677f-455f-a24c-e64eef9cdb8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:34:55.001942 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 792/1564] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:34:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:34:55.002800 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ad921f-677f-455f-a24c-e64eef9cdb8c None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:34:55.011855 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ad921f-677f-455f-a24c-e64eef9cdb8c None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:34:55.016491 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adb2332-86a7-4d1d-96a6-5c3fef284e4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:34:55.017110 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adb2332-86a7-4d1d-96a6-5c3fef284e4c admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:34:55.017399 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adb2332-86a7-4d1d-96a6-5c3fef284e4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:34:55.017703 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adb2332-86a7-4d1d-96a6-5c3fef284e4c admin admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:34:55.018439 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adb2332-86a7-4d1d-96a6-5c3fef284e4c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:34:55.018741 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adb2332-86a7-4d1d-96a6-5c3fef284e4c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:34:55.020361 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adb2332-86a7-4d1d-96a6-5c3fef284e4c admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:34:55.032659 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 793/1565] 10.222.0.66 () {58 vars in 1152 bytes} [Mon Jul 18 02:34:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:34:55.040350 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 773/1566] 10.222.0.66 () {60 vars in 1143 bytes} [Mon Jul 18 02:34:54 2022] POST /identity/v3/users => generated 289 bytes in 342 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:34:55.040957 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adb2332-86a7-4d1d-96a6-5c3fef284e4c admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:34:55.041078 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c285c6da-967c-4f01-9813-9ab9bbc98356 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:34:55.049193 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c285c6da-967c-4f01-9813-9ab9bbc98356 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:34:55.054803 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adb2332-86a7-4d1d-96a6-5c3fef284e4c admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:34:55.055037 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659f4ea3-bffd-40bd-b862-9b1079b563ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:34:55.055037 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659f4ea3-bffd-40bd-b862-9b1079b563ee admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:34:55.055037 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659f4ea3-bffd-40bd-b862-9b1079b563ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:34:55.055329 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659f4ea3-bffd-40bd-b862-9b1079b563ee admin admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:34:55.056002 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659f4ea3-bffd-40bd-b862-9b1079b563ee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:34:55.056319 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659f4ea3-bffd-40bd-b862-9b1079b563ee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:34:55.057930 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659f4ea3-bffd-40bd-b862-9b1079b563ee admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:34:55.058423 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3f2ff8-3f82-41b3-9d3f-05d837f24f30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:34:55.059085 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3f2ff8-3f82-41b3-9d3f-05d837f24f30 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:34:55.059368 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3f2ff8-3f82-41b3-9d3f-05d837f24f30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:34:55.059644 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3f2ff8-3f82-41b3-9d3f-05d837f24f30 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:34:55.060359 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3f2ff8-3f82-41b3-9d3f-05d837f24f30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:34:55.061953 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3f2ff8-3f82-41b3-9d3f-05d837f24f30 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:34:55.070603 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 774/1567] 10.222.0.66 () {58 vars in 1152 bytes} [Mon Jul 18 02:34:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:34:55.071330 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659f4ea3-bffd-40bd-b862-9b1079b563ee admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:34:55.075518 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 794/1568] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:34:55 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:34:55.075518 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3f2ff8-3f82-41b3-9d3f-05d837f24f30 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:34:55.087316 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659f4ea3-bffd-40bd-b862-9b1079b563ee admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:34:55.087489 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3f2ff8-3f82-41b3-9d3f-05d837f24f30 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:34:55.091802 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16eeb836-78ee-4b42-bd9e-b0fa68ec2272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:34:55.092022 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a31a41-7a60-4abb-b5a3-bbedf95d5202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:34:55.092586 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eeb836-78ee-4b42-bd9e-b0fa68ec2272 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:34:55.092713 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a31a41-7a60-4abb-b5a3-bbedf95d5202 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:34:55.092871 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eeb836-78ee-4b42-bd9e-b0fa68ec2272 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:34:55.092979 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a31a41-7a60-4abb-b5a3-bbedf95d5202 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:34:55.093149 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eeb836-78ee-4b42-bd9e-b0fa68ec2272 admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:34:55.093347 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a31a41-7a60-4abb-b5a3-bbedf95d5202 admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:34:55.093832 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16eeb836-78ee-4b42-bd9e-b0fa68ec2272 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:34:55.094478 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a31a41-7a60-4abb-b5a3-bbedf95d5202 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:34:55.096283 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a31a41-7a60-4abb-b5a3-bbedf95d5202 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:34:55.096629 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16eeb836-78ee-4b42-bd9e-b0fa68ec2272 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:34:55.242390 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 775/1569] 10.222.0.66 () {60 vars in 1143 bytes} [Mon Jul 18 02:34:55 2022] POST /identity/v3/users => generated 284 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:34:55.242390 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16eeb836-78ee-4b42-bd9e-b0fa68ec2272 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:34:55.242723 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 795/1570] 10.222.0.66 () {60 vars in 1152 bytes} [Mon Jul 18 02:34:55 2022] POST /identity/v3/projects => generated 383 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:34:55.242723 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a31a41-7a60-4abb-b5a3-bbedf95d5202 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:34:55.250313 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16eeb836-78ee-4b42-bd9e-b0fa68ec2272 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:34:55.307264 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a31a41-7a60-4abb-b5a3-bbedf95d5202 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:34:55.308273 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556827dc-8e9f-480b-97e3-2c7238079af2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:34:55.309010 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556827dc-8e9f-480b-97e3-2c7238079af2 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:34:55.309232 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556827dc-8e9f-480b-97e3-2c7238079af2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:34:55.309521 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556827dc-8e9f-480b-97e3-2c7238079af2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:34:55.310275 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556827dc-8e9f-480b-97e3-2c7238079af2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:34:55.310586 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556827dc-8e9f-480b-97e3-2c7238079af2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:34:55.311983 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4b3740-ed5b-40f6-a9fe-833745fbe019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:34:55.312207 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556827dc-8e9f-480b-97e3-2c7238079af2 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:34:55.312644 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4b3740-ed5b-40f6-a9fe-833745fbe019 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:34:55.312964 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4b3740-ed5b-40f6-a9fe-833745fbe019 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:34:55.313235 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4b3740-ed5b-40f6-a9fe-833745fbe019 admin admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:34:55.313951 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4b3740-ed5b-40f6-a9fe-833745fbe019 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:34:55.314396 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4b3740-ed5b-40f6-a9fe-833745fbe019 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:34:55.315984 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4b3740-ed5b-40f6-a9fe-833745fbe019 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:34:55.325318 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 776/1571] 10.222.0.66 () {58 vars in 1152 bytes} [Mon Jul 18 02:34:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:34:55.326405 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556827dc-8e9f-480b-97e3-2c7238079af2 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:34:55.328152 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 796/1572] 10.222.0.66 () {58 vars in 1152 bytes} [Mon Jul 18 02:34:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:34:55.331007 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4b3740-ed5b-40f6-a9fe-833745fbe019 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:34:55.338702 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556827dc-8e9f-480b-97e3-2c7238079af2 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:34:55.341302 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4b3740-ed5b-40f6-a9fe-833745fbe019 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:34:55.343475 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf423ab-de0a-4991-b785-17a6cb816a35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:34:55.344754 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf423ab-de0a-4991-b785-17a6cb816a35 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:34:55.345275 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf423ab-de0a-4991-b785-17a6cb816a35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:34:55.345601 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf423ab-de0a-4991-b785-17a6cb816a35 admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:34:55.346348 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf423ab-de0a-4991-b785-17a6cb816a35 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:34:55.346836 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3d037f-c820-4281-8914-f26298b3b4d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:34:55.347491 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3d037f-c820-4281-8914-f26298b3b4d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:34:55.347782 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3d037f-c820-4281-8914-f26298b3b4d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:34:55.348059 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3d037f-c820-4281-8914-f26298b3b4d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:34:55.348255 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf423ab-de0a-4991-b785-17a6cb816a35 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:34:55.348778 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3d037f-c820-4281-8914-f26298b3b4d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:34:55.350407 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3d037f-c820-4281-8914-f26298b3b4d3 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:34:55.360925 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 797/1573] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:34:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:34:55.361555 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3d037f-c820-4281-8914-f26298b3b4d3 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:34:55.366613 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:34:55.371800 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3d037f-c820-4281-8914-f26298b3b4d3 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:34:55.376662 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380ef1f9-7753-4c8b-b3c6-a3551cad7e71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:34:55.377533 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380ef1f9-7753-4c8b-b3c6-a3551cad7e71 admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:34:55.377817 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380ef1f9-7753-4c8b-b3c6-a3551cad7e71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:34:55.378110 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380ef1f9-7753-4c8b-b3c6-a3551cad7e71 admin admin] PATH_INFO: `/v3/projects/805769e9d58340d48402101bf7eb967e/users/253d2bf6c1aa42e6a55a760e5e37f01a/roles/65feabf3b27944a8b01a86c8ac608457` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:34:55.394193 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380ef1f9-7753-4c8b-b3c6-a3551cad7e71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=805769e9d58340d48402101bf7eb967e, user_id=253d2bf6c1aa42e6a55a760e5e37f01a, role_id=65feabf3b27944a8b01a86c8ac608457)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:34:55.396859 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380ef1f9-7753-4c8b-b3c6-a3551cad7e71 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:34:55.434003 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 777/1574] 10.222.0.66 () {60 vars in 1152 bytes} [Mon Jul 18 02:34:55 2022] POST /identity/v3/projects => generated 383 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:34:55.435085 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf423ab-de0a-4991-b785-17a6cb816a35 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:34:55.495566 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf423ab-de0a-4991-b785-17a6cb816a35 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:34:55.500005 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1488c36f-929a-4097-a13c-fdab38e1a741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:34:55.501068 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1488c36f-929a-4097-a13c-fdab38e1a741 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:34:55.501360 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1488c36f-929a-4097-a13c-fdab38e1a741 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:34:55.501637 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1488c36f-929a-4097-a13c-fdab38e1a741 admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:34:55.502353 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1488c36f-929a-4097-a13c-fdab38e1a741 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:34:55.504255 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1488c36f-929a-4097-a13c-fdab38e1a741 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:34:55.512497 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 798/1575] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:34:55 2022] PUT /identity/v3/projects/805769e9d58340d48402101bf7eb967e/users/253d2bf6c1aa42e6a55a760e5e37f01a/roles/65feabf3b27944a8b01a86c8ac608457 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:34:55.513409 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380ef1f9-7753-4c8b-b3c6-a3551cad7e71 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:34:55.574523 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380ef1f9-7753-4c8b-b3c6-a3551cad7e71 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:34:55.579727 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a2d9a5-9ec4-4e07-a634-137ce30e7f80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:34:55.580385 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a2d9a5-9ec4-4e07-a634-137ce30e7f80 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:34:55.580672 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a2d9a5-9ec4-4e07-a634-137ce30e7f80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:34:55.580951 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a2d9a5-9ec4-4e07-a634-137ce30e7f80 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:34:55.581658 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a2d9a5-9ec4-4e07-a634-137ce30e7f80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:34:55.583305 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a2d9a5-9ec4-4e07-a634-137ce30e7f80 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:34:55.588402 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 778/1576] 10.222.0.66 () {60 vars in 1152 bytes} [Mon Jul 18 02:34:55 2022] POST /identity/v3/projects => generated 387 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:34:55.589285 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1488c36f-929a-4097-a13c-fdab38e1a741 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:34:55.597532 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 799/1577] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:34:55 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:34:55.598628 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a2d9a5-9ec4-4e07-a634-137ce30e7f80 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:34:55.612648 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a2d9a5-9ec4-4e07-a634-137ce30e7f80 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:34:55.613446 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:34:55.656216 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1488c36f-929a-4097-a13c-fdab38e1a741 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:34:55.661469 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557e584c-39e9-4e00-8e63-90f51963c953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:34:55.662154 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557e584c-39e9-4e00-8e63-90f51963c953 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:34:55.662437 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557e584c-39e9-4e00-8e63-90f51963c953 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:34:55.662713 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557e584c-39e9-4e00-8e63-90f51963c953 admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:34:55.663483 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557e584c-39e9-4e00-8e63-90f51963c953 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:34:55.665405 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557e584c-39e9-4e00-8e63-90f51963c953 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:34:55.715692 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357a59ef-ea0d-4cfc-b544-0beb7159729b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:34:55.716354 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357a59ef-ea0d-4cfc-b544-0beb7159729b admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:34:55.716640 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357a59ef-ea0d-4cfc-b544-0beb7159729b admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:34:55.716920 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357a59ef-ea0d-4cfc-b544-0beb7159729b admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:34:55.717664 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357a59ef-ea0d-4cfc-b544-0beb7159729b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:34:55.720266 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357a59ef-ea0d-4cfc-b544-0beb7159729b admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:34:55.773666 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 779/1578] 10.222.0.66 () {60 vars in 1143 bytes} [Mon Jul 18 02:34:55 2022] POST /identity/v3/users => generated 292 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:34:55.774424 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557e584c-39e9-4e00-8e63-90f51963c953 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:34:55.786311 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557e584c-39e9-4e00-8e63-90f51963c953 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:34:55.791089 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6e2639-7849-4e48-bb16-53c15bea0d09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:34:55.791737 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6e2639-7849-4e48-bb16-53c15bea0d09 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:34:55.792027 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6e2639-7849-4e48-bb16-53c15bea0d09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:34:55.792307 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6e2639-7849-4e48-bb16-53c15bea0d09 admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:34:55.792999 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6e2639-7849-4e48-bb16-53c15bea0d09 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:34:55.795049 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6e2639-7849-4e48-bb16-53c15bea0d09 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:34:55.804750 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 800/1579] 10.222.0.66 () {60 vars in 1152 bytes} [Mon Jul 18 02:34:55 2022] POST /identity/v3/projects => generated 403 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:34:55.805594 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357a59ef-ea0d-4cfc-b544-0beb7159729b admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:34:55.874367 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357a59ef-ea0d-4cfc-b544-0beb7159729b admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:34:55.879106 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2672bd9-ccd1-4f47-a8f1-e0d661055be1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:34:55.879767 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2672bd9-ccd1-4f47-a8f1-e0d661055be1 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:34:55.880055 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2672bd9-ccd1-4f47-a8f1-e0d661055be1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:34:55.880332 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2672bd9-ccd1-4f47-a8f1-e0d661055be1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:34:55.881073 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2672bd9-ccd1-4f47-a8f1-e0d661055be1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:34:55.882713 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2672bd9-ccd1-4f47-a8f1-e0d661055be1 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:34:55.883962 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:34:55.893204 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 801/1580] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:34:55 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:34:55.909309 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2672bd9-ccd1-4f47-a8f1-e0d661055be1 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:34:55.909463 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 780/1581] 10.222.0.66 () {60 vars in 1152 bytes} [Mon Jul 18 02:34:55 2022] POST /identity/v3/projects => generated 377 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:34:55.909463 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1fb170-6592-4f40-961f-1db458af077f None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:34:55.909463 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1fb170-6592-4f40-961f-1db458af077f None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:34:55.909463 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1fb170-6592-4f40-961f-1db458af077f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:34:55.909862 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2672bd9-ccd1-4f47-a8f1-e0d661055be1 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:34:55.926829 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-eb1fb170-6592-4f40-961f-1db458af077f None None] MFA Rules not processed for user `253d2bf6c1aa42e6a55a760e5e37f01a`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:34:55.932297 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-eb1fb170-6592-4f40-961f-1db458af077f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:34:55.964506 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb198cad-ce08-4334-a564-89e37ef32a2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:34:55.965189 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb198cad-ce08-4334-a564-89e37ef32a2e admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:34:55.965471 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb198cad-ce08-4334-a564-89e37ef32a2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:34:55.965742 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb198cad-ce08-4334-a564-89e37ef32a2e admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:34:55.966489 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb198cad-ce08-4334-a564-89e37ef32a2e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:34:55.968485 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb198cad-ce08-4334-a564-89e37ef32a2e admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:34:55.994035 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 781/1582] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:34:55 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:34:55.995105 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1fb170-6592-4f40-961f-1db458af077f None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:34:56.005336 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1fb170-6592-4f40-961f-1db458af077f None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:34:56.009889 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9599c4-9c34-4ffd-bfd3-4147258021d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:34:56.010563 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9599c4-9c34-4ffd-bfd3-4147258021d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:34:56.010850 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9599c4-9c34-4ffd-bfd3-4147258021d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:34:56.011129 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9599c4-9c34-4ffd-bfd3-4147258021d0 admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:34:56.011831 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9599c4-9c34-4ffd-bfd3-4147258021d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:34:56.013753 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9599c4-9c34-4ffd-bfd3-4147258021d0 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:34:56.059700 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 802/1583] 10.222.0.66 () {60 vars in 1143 bytes} [Mon Jul 18 02:34:55 2022] POST /identity/v3/users => generated 292 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:34:56.060636 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb198cad-ce08-4334-a564-89e37ef32a2e admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:34:56.070610 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb198cad-ce08-4334-a564-89e37ef32a2e admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:34:56.075150 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a603e75a-6991-450c-8c72-b01eaf37f659 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:34:56.075779 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a603e75a-6991-450c-8c72-b01eaf37f659 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:34:56.076060 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a603e75a-6991-450c-8c72-b01eaf37f659 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:34:56.076335 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a603e75a-6991-450c-8c72-b01eaf37f659 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:34:56.077046 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a603e75a-6991-450c-8c72-b01eaf37f659 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:34:56.079227 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a603e75a-6991-450c-8c72-b01eaf37f659 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:34:56.083179 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 782/1584] 10.222.0.66 () {60 vars in 1143 bytes} [Mon Jul 18 02:34:55 2022] POST /identity/v3/users => generated 294 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:34:56.088968 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9599c4-9c34-4ffd-bfd3-4147258021d0 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:34:56.089388 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 803/1585] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:34:56 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:34:56.090252 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a603e75a-6991-450c-8c72-b01eaf37f659 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:34:56.097189 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9599c4-9c34-4ffd-bfd3-4147258021d0 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:34:56.101995 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1f49f3-34cd-467d-9535-b0fc256a790f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:34:56.102370 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a603e75a-6991-450c-8c72-b01eaf37f659 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:34:56.102705 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1f49f3-34cd-467d-9535-b0fc256a790f admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:34:56.102993 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1f49f3-34cd-467d-9535-b0fc256a790f admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:34:56.103272 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1f49f3-34cd-467d-9535-b0fc256a790f admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:34:56.104094 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1f49f3-34cd-467d-9535-b0fc256a790f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:34:56.105699 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1f49f3-34cd-467d-9535-b0fc256a790f admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:34:56.107121 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73accf7b-07f4-4851-8c40-3dbe6b5cfd33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:34:56.107768 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73accf7b-07f4-4851-8c40-3dbe6b5cfd33 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:34:56.108053 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73accf7b-07f4-4851-8c40-3dbe6b5cfd33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:34:56.108360 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73accf7b-07f4-4851-8c40-3dbe6b5cfd33 admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:34:56.109025 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73accf7b-07f4-4851-8c40-3dbe6b5cfd33 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:34:56.111017 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73accf7b-07f4-4851-8c40-3dbe6b5cfd33 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:34:56.116776 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 783/1586] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:34:56 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:34:56.117549 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1f49f3-34cd-467d-9535-b0fc256a790f admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:34:56.127372 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1f49f3-34cd-467d-9535-b0fc256a790f admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:34:56.132037 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca51b39-7319-4a01-a2c9-13dcdc9e3716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:34:56.134877 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca51b39-7319-4a01-a2c9-13dcdc9e3716 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:34:56.134877 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca51b39-7319-4a01-a2c9-13dcdc9e3716 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:34:56.134877 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca51b39-7319-4a01-a2c9-13dcdc9e3716 admin admin] PATH_INFO: `/v3/projects/85204e1ae6844b70b45f8452887d5385/users/879f7a4cf6664e76bddffc804658b250/roles/65feabf3b27944a8b01a86c8ac608457` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:34:56.148747 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca51b39-7319-4a01-a2c9-13dcdc9e3716 admin admin] RBAC: Authorizing `identity:create_grant(project_id=85204e1ae6844b70b45f8452887d5385, user_id=879f7a4cf6664e76bddffc804658b250, role_id=65feabf3b27944a8b01a86c8ac608457)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:34:56.149874 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: WARNING py.warnings [None req-2ca51b39-7319-4a01-a2c9-13dcdc9e3716 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 02:34:56.149874 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: warnings.warn(msg) Jul 18 02:34:56.149874 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: Jul 18 02:34:56.151973 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca51b39-7319-4a01-a2c9-13dcdc9e3716 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:34:56.180112 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 804/1587] 10.222.0.66 () {60 vars in 1143 bytes} [Mon Jul 18 02:34:56 2022] POST /identity/v3/users => generated 302 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:34:56.181077 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73accf7b-07f4-4851-8c40-3dbe6b5cfd33 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:34:56.190573 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73accf7b-07f4-4851-8c40-3dbe6b5cfd33 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:34:56.195214 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff54bdf8-fc51-4841-b4cc-ab82eacc3165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:34:56.195872 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff54bdf8-fc51-4841-b4cc-ab82eacc3165 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:34:56.196154 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff54bdf8-fc51-4841-b4cc-ab82eacc3165 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:34:56.196481 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff54bdf8-fc51-4841-b4cc-ab82eacc3165 admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:34:56.197158 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff54bdf8-fc51-4841-b4cc-ab82eacc3165 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:34:56.199110 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff54bdf8-fc51-4841-b4cc-ab82eacc3165 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:34:56.249952 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 784/1588] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:34:56 2022] PUT /identity/v3/projects/85204e1ae6844b70b45f8452887d5385/users/879f7a4cf6664e76bddffc804658b250/roles/65feabf3b27944a8b01a86c8ac608457 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:34:56.250940 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca51b39-7319-4a01-a2c9-13dcdc9e3716 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:34:56.276989 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:34:56.278386 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 805/1589] 10.222.0.66 () {60 vars in 1143 bytes} [Mon Jul 18 02:34:56 2022] POST /identity/v3/users => generated 289 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:34:56.283141 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff54bdf8-fc51-4841-b4cc-ab82eacc3165 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:34:56.293003 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff54bdf8-fc51-4841-b4cc-ab82eacc3165 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:34:56.314763 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca51b39-7319-4a01-a2c9-13dcdc9e3716 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:34:56.319419 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912599f0-036a-4b3d-873a-dfe623797c9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:34:56.320076 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912599f0-036a-4b3d-873a-dfe623797c9d admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:34:56.320356 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912599f0-036a-4b3d-873a-dfe623797c9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:34:56.320631 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912599f0-036a-4b3d-873a-dfe623797c9d admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:34:56.321318 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912599f0-036a-4b3d-873a-dfe623797c9d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:34:56.323254 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912599f0-036a-4b3d-873a-dfe623797c9d admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:34:56.352030 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac0237b-6667-46a0-a55b-7629d3c1f0c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:34:56.352703 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac0237b-6667-46a0-a55b-7629d3c1f0c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:34:56.352988 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac0237b-6667-46a0-a55b-7629d3c1f0c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:34:56.353257 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac0237b-6667-46a0-a55b-7629d3c1f0c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:34:56.354001 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac0237b-6667-46a0-a55b-7629d3c1f0c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:34:56.356227 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac0237b-6667-46a0-a55b-7629d3c1f0c9 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:34:56.366265 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 806/1590] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:34:56 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:34:56.367045 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac0237b-6667-46a0-a55b-7629d3c1f0c9 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:34:56.376705 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac0237b-6667-46a0-a55b-7629d3c1f0c9 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:34:56.381223 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7fc95d-4647-4fdf-af3e-1fb3cf963131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:34:56.386389 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7fc95d-4647-4fdf-af3e-1fb3cf963131 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:34:56.386389 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7fc95d-4647-4fdf-af3e-1fb3cf963131 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:34:56.386389 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7fc95d-4647-4fdf-af3e-1fb3cf963131 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:34:56.386389 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7fc95d-4647-4fdf-af3e-1fb3cf963131 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:34:56.386389 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7fc95d-4647-4fdf-af3e-1fb3cf963131 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:34:56.395370 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 807/1591] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:34:56 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:34:56.396329 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7fc95d-4647-4fdf-af3e-1fb3cf963131 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:34:56.406240 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7fc95d-4647-4fdf-af3e-1fb3cf963131 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:34:56.411657 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3904d4fc-d6b7-4de2-a0d8-ba60061dabf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:34:56.412511 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3904d4fc-d6b7-4de2-a0d8-ba60061dabf9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:34:56.412795 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3904d4fc-d6b7-4de2-a0d8-ba60061dabf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:34:56.413070 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3904d4fc-d6b7-4de2-a0d8-ba60061dabf9 admin admin] PATH_INFO: `/v3/projects/117f57edd1524d03b049a87c6b8bd946/users/e0bb4bfb06ec4eae87355c6e84328c26/roles/65feabf3b27944a8b01a86c8ac608457` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:34:56.427931 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3904d4fc-d6b7-4de2-a0d8-ba60061dabf9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=117f57edd1524d03b049a87c6b8bd946, user_id=e0bb4bfb06ec4eae87355c6e84328c26, role_id=65feabf3b27944a8b01a86c8ac608457)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:34:56.432321 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 785/1592] 10.222.0.66 () {60 vars in 1152 bytes} [Mon Jul 18 02:34:56 2022] POST /identity/v3/projects => generated 355 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:34:56.432321 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912599f0-036a-4b3d-873a-dfe623797c9d admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:34:56.432612 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3904d4fc-d6b7-4de2-a0d8-ba60061dabf9 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:34:56.492404 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912599f0-036a-4b3d-873a-dfe623797c9d admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:34:56.497150 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9737b73-bcaf-4729-bcc9-48f0d69cf2fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:34:56.497809 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9737b73-bcaf-4729-bcc9-48f0d69cf2fb admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:34:56.498120 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9737b73-bcaf-4729-bcc9-48f0d69cf2fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:34:56.498395 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9737b73-bcaf-4729-bcc9-48f0d69cf2fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:34:56.499107 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9737b73-bcaf-4729-bcc9-48f0d69cf2fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:34:56.501254 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9737b73-bcaf-4729-bcc9-48f0d69cf2fb admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:34:56.511027 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 786/1593] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:34:56 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:34:56.512731 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9737b73-bcaf-4729-bcc9-48f0d69cf2fb admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:34:56.515495 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:34:56.522030 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 808/1594] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:34:56 2022] PUT /identity/v3/projects/117f57edd1524d03b049a87c6b8bd946/users/e0bb4bfb06ec4eae87355c6e84328c26/roles/65feabf3b27944a8b01a86c8ac608457 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:34:56.523018 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9737b73-bcaf-4729-bcc9-48f0d69cf2fb admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:34:56.523419 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a01455-bc01-4047-a01a-e18bcb4030dc None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:34:56.523716 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a01455-bc01-4047-a01a-e18bcb4030dc None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:34:56.524146 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a01455-bc01-4047-a01a-e18bcb4030dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:34:56.541180 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-29a01455-bc01-4047-a01a-e18bcb4030dc None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:34:56.542443 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-29a01455-bc01-4047-a01a-e18bcb4030dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:34:56.582976 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08721b1-6a30-4f37-bc83-9c767f52b879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:34:56.583693 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08721b1-6a30-4f37-bc83-9c767f52b879 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:34:56.583980 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08721b1-6a30-4f37-bc83-9c767f52b879 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:34:56.584260 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08721b1-6a30-4f37-bc83-9c767f52b879 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:34:56.585024 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08721b1-6a30-4f37-bc83-9c767f52b879 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:34:56.586697 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08721b1-6a30-4f37-bc83-9c767f52b879 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:34:56.597168 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 787/1595] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:34:56 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:34:56.598402 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08721b1-6a30-4f37-bc83-9c767f52b879 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:34:56.603745 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 809/1596] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:34:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:34:56.611090 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a01455-bc01-4047-a01a-e18bcb4030dc None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:34:56.617605 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a01455-bc01-4047-a01a-e18bcb4030dc None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:34:56.618057 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08721b1-6a30-4f37-bc83-9c767f52b879 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:34:56.622211 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab3bb52-70c4-4587-98c1-7827a6629c41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:34:56.622678 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f443bae-c068-496c-ba5e-d9bbb8cb0414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:34:56.622931 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab3bb52-70c4-4587-98c1-7827a6629c41 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:34:56.623216 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab3bb52-70c4-4587-98c1-7827a6629c41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:34:56.623339 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f443bae-c068-496c-ba5e-d9bbb8cb0414 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:34:56.623503 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab3bb52-70c4-4587-98c1-7827a6629c41 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:34:56.623607 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f443bae-c068-496c-ba5e-d9bbb8cb0414 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:34:56.623877 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f443bae-c068-496c-ba5e-d9bbb8cb0414 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:34:56.624204 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab3bb52-70c4-4587-98c1-7827a6629c41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:34:56.624599 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f443bae-c068-496c-ba5e-d9bbb8cb0414 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:34:56.626466 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab3bb52-70c4-4587-98c1-7827a6629c41 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:34:56.626873 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f443bae-c068-496c-ba5e-d9bbb8cb0414 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:34:56.636690 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 810/1597] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:34:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:34:56.637601 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab3bb52-70c4-4587-98c1-7827a6629c41 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:34:56.637751 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 788/1598] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:34:56 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:34:56.638298 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f443bae-c068-496c-ba5e-d9bbb8cb0414 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:34:56.648858 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f443bae-c068-496c-ba5e-d9bbb8cb0414 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:34:56.650822 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab3bb52-70c4-4587-98c1-7827a6629c41 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:34:56.653452 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c992b76-3787-4cfc-ab2d-1d6151244c93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:34:56.654108 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c992b76-3787-4cfc-ab2d-1d6151244c93 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:34:56.654411 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c992b76-3787-4cfc-ab2d-1d6151244c93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:34:56.654686 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c992b76-3787-4cfc-ab2d-1d6151244c93 admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:34:56.655391 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c992b76-3787-4cfc-ab2d-1d6151244c93 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:34:56.655632 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2884ba01-ce59-453c-ab6b-11dca6cc6aa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:34:56.656370 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2884ba01-ce59-453c-ab6b-11dca6cc6aa6 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:34:56.656734 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2884ba01-ce59-453c-ab6b-11dca6cc6aa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:34:56.657015 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2884ba01-ce59-453c-ab6b-11dca6cc6aa6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:34:56.657333 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c992b76-3787-4cfc-ab2d-1d6151244c93 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:34:56.657769 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2884ba01-ce59-453c-ab6b-11dca6cc6aa6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:34:56.659426 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2884ba01-ce59-453c-ab6b-11dca6cc6aa6 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:34:56.669748 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 811/1599] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:34:56 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:34:56.670818 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2884ba01-ce59-453c-ab6b-11dca6cc6aa6 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:34:56.680488 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2884ba01-ce59-453c-ab6b-11dca6cc6aa6 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:34:56.685182 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22822c2d-083e-45a9-a4c6-6b7985b41321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:34:56.686061 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22822c2d-083e-45a9-a4c6-6b7985b41321 admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:34:56.686385 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22822c2d-083e-45a9-a4c6-6b7985b41321 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:34:56.686667 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22822c2d-083e-45a9-a4c6-6b7985b41321 admin admin] PATH_INFO: `/v3/projects/4b02a417904a474abcab1eec5cea9c7e/users/94c16391d39f4498a55c89bf2838788f/roles/65feabf3b27944a8b01a86c8ac608457` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:34:56.701767 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22822c2d-083e-45a9-a4c6-6b7985b41321 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b02a417904a474abcab1eec5cea9c7e, user_id=94c16391d39f4498a55c89bf2838788f, role_id=65feabf3b27944a8b01a86c8ac608457)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:34:56.705215 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22822c2d-083e-45a9-a4c6-6b7985b41321 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:34:56.769261 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 789/1600] 10.222.0.66 () {60 vars in 1143 bytes} [Mon Jul 18 02:34:56 2022] POST /identity/v3/users => generated 284 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:34:56.770715 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2651a2a8-2a64-449b-b923-f8d3764c0972 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:34:56.771006 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2651a2a8-2a64-449b-b923-f8d3764c0972 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:34:56.771286 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2651a2a8-2a64-449b-b923-f8d3764c0972 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:34:56.789630 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:34:56.797376 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-2651a2a8-2a64-449b-b923-f8d3764c0972 None None] MFA Rules not processed for user `e0bb4bfb06ec4eae87355c6e84328c26`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:34:56.798223 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 812/1601] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:34:56 2022] PUT /identity/v3/projects/4b02a417904a474abcab1eec5cea9c7e/users/94c16391d39f4498a55c89bf2838788f/roles/65feabf3b27944a8b01a86c8ac608457 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:34:56.798900 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-2651a2a8-2a64-449b-b923-f8d3764c0972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:34:56.802733 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22822c2d-083e-45a9-a4c6-6b7985b41321 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:34:56.862593 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22822c2d-083e-45a9-a4c6-6b7985b41321 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:34:56.863420 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 790/1602] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:34:56 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:34:56.867563 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca331c9f-92e3-4c0e-b2e2-f26380ab57f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:34:56.868031 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2651a2a8-2a64-449b-b923-f8d3764c0972 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:34:56.868561 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca331c9f-92e3-4c0e-b2e2-f26380ab57f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:34:56.868917 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca331c9f-92e3-4c0e-b2e2-f26380ab57f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:34:56.869533 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca331c9f-92e3-4c0e-b2e2-f26380ab57f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:34:56.869987 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca331c9f-92e3-4c0e-b2e2-f26380ab57f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:34:56.871643 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca331c9f-92e3-4c0e-b2e2-f26380ab57f9 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:34:56.883279 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 813/1603] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:34:56 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:34:56.884424 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca331c9f-92e3-4c0e-b2e2-f26380ab57f9 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:34:56.895153 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca331c9f-92e3-4c0e-b2e2-f26380ab57f9 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:34:56.900153 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a89c887-8248-4b88-85a2-2a0914084663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:34:56.902425 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a89c887-8248-4b88-85a2-2a0914084663 admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:34:56.902425 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a89c887-8248-4b88-85a2-2a0914084663 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:34:56.902425 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a89c887-8248-4b88-85a2-2a0914084663 admin admin] PATH_INFO: `/v3/projects/5a76c0d5639d42c9bb2387d75f4b0f95/users/932c4d9cb0034247905e0bc72b6ccbce/roles/65feabf3b27944a8b01a86c8ac608457` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:34:56.917632 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a89c887-8248-4b88-85a2-2a0914084663 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a76c0d5639d42c9bb2387d75f4b0f95, user_id=932c4d9cb0034247905e0bc72b6ccbce, role_id=65feabf3b27944a8b01a86c8ac608457)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:34:56.920501 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a89c887-8248-4b88-85a2-2a0914084663 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:34:56.926864 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2651a2a8-2a64-449b-b923-f8d3764c0972 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:34:56.930462 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a3cf67-bb91-461f-a7b1-bb0e64a5dd3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eaf19a259c2741cea375a38763e33cdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:34:56.931103 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a3cf67-bb91-461f-a7b1-bb0e64a5dd3a service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:34:56.931426 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a3cf67-bb91-461f-a7b1-bb0e64a5dd3a service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:34:56.936745 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a3cf67-bb91-461f-a7b1-bb0e64a5dd3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:34:56.940884 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a3cf67-bb91-461f-a7b1-bb0e64a5dd3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:34:56.942363 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a3cf67-bb91-461f-a7b1-bb0e64a5dd3a service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:34:56.951762 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 791/1604] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:34:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:34:56.952527 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a3cf67-bb91-461f-a7b1-bb0e64a5dd3a service neutron] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:34:56.962257 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a3cf67-bb91-461f-a7b1-bb0e64a5dd3a service neutron] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:34:56.967093 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634ab025-03b8-488d-8f09-2bf6e7cf3a29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:34:56.969869 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634ab025-03b8-488d-8f09-2bf6e7cf3a29 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:34:56.969869 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634ab025-03b8-488d-8f09-2bf6e7cf3a29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:34:56.969869 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634ab025-03b8-488d-8f09-2bf6e7cf3a29 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:34:56.970458 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634ab025-03b8-488d-8f09-2bf6e7cf3a29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:34:56.972407 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634ab025-03b8-488d-8f09-2bf6e7cf3a29 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:34:56.982429 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 792/1605] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:34:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:34:56.983184 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634ab025-03b8-488d-8f09-2bf6e7cf3a29 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:34:56.992188 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 814/1606] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:34:56 2022] PUT /identity/v3/projects/5a76c0d5639d42c9bb2387d75f4b0f95/users/932c4d9cb0034247905e0bc72b6ccbce/roles/65feabf3b27944a8b01a86c8ac608457 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:34:56.993168 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a89c887-8248-4b88-85a2-2a0914084663 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:34:56.993608 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634ab025-03b8-488d-8f09-2bf6e7cf3a29 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:34:57.025650 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:34:57.058987 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a89c887-8248-4b88-85a2-2a0914084663 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:34:57.059468 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa01aa0-1d9a-4cb9-bf8f-f3fdfb7778d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:34:57.060370 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa01aa0-1d9a-4cb9-bf8f-f3fdfb7778d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:34:57.060653 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa01aa0-1d9a-4cb9-bf8f-f3fdfb7778d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:34:57.060928 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa01aa0-1d9a-4cb9-bf8f-f3fdfb7778d1 admin admin] PATH_INFO: `/v3/projects/f3c1745729fe47238223557221c2e905/users/7ddb197852db4b20a84aa6c0862468ef/roles/65feabf3b27944a8b01a86c8ac608457` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:34:57.065414 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b581713-1784-4d22-ab08-39bde0b9eada admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:34:57.066095 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b581713-1784-4d22-ab08-39bde0b9eada admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:34:57.066398 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b581713-1784-4d22-ab08-39bde0b9eada admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:34:57.066679 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b581713-1784-4d22-ab08-39bde0b9eada admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:34:57.067433 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b581713-1784-4d22-ab08-39bde0b9eada admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:34:57.069061 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b581713-1784-4d22-ab08-39bde0b9eada admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:34:57.076292 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa01aa0-1d9a-4cb9-bf8f-f3fdfb7778d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3c1745729fe47238223557221c2e905, user_id=7ddb197852db4b20a84aa6c0862468ef, role_id=65feabf3b27944a8b01a86c8ac608457)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:34:57.079239 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa01aa0-1d9a-4cb9-bf8f-f3fdfb7778d1 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:34:57.083739 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 815/1607] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:34:56 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:34:57.083739 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1b2742-8239-4a25-9458-5b53af24e3d5 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:34:57.083739 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1b2742-8239-4a25-9458-5b53af24e3d5 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:34:57.083739 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1b2742-8239-4a25-9458-5b53af24e3d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:34:57.101005 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-5d1b2742-8239-4a25-9458-5b53af24e3d5 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:34:57.103084 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-5d1b2742-8239-4a25-9458-5b53af24e3d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:34:57.110940 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 816/1608] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:34:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:34:57.111809 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1b2742-8239-4a25-9458-5b53af24e3d5 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:34:57.169970 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1b2742-8239-4a25-9458-5b53af24e3d5 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:34:57.173840 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1225b5dc-c692-409c-807d-c0327b5805f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd49388facdf446885e55b45ef91f01e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:34:57.174547 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1225b5dc-c692-409c-807d-c0327b5805f7 service cinder] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:34:57.174831 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1225b5dc-c692-409c-807d-c0327b5805f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:34:57.175104 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1225b5dc-c692-409c-807d-c0327b5805f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:34:57.191376 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 793/1609] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:34:56 2022] PUT /identity/v3/projects/f3c1745729fe47238223557221c2e905/users/7ddb197852db4b20a84aa6c0862468ef/roles/65feabf3b27944a8b01a86c8ac608457 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:34:57.192191 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa01aa0-1d9a-4cb9-bf8f-f3fdfb7778d1 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:34:57.212132 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:34:57.233893 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1225b5dc-c692-409c-807d-c0327b5805f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:34:57.235410 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1225b5dc-c692-409c-807d-c0327b5805f7 service cinder] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:34:57.244777 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 817/1610] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:34:57 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:34:57.246131 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1225b5dc-c692-409c-807d-c0327b5805f7 service cinder] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:34:57.256384 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1225b5dc-c692-409c-807d-c0327b5805f7 service cinder] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:34:57.259018 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa01aa0-1d9a-4cb9-bf8f-f3fdfb7778d1 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:34:57.261817 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adfd2d1-90ac-4824-968d-662a2af0f8dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:34:57.262760 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adfd2d1-90ac-4824-968d-662a2af0f8dd admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:34:57.263022 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adfd2d1-90ac-4824-968d-662a2af0f8dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:34:57.263299 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adfd2d1-90ac-4824-968d-662a2af0f8dd admin admin] PATH_INFO: `/v3/projects/d0695b3617d845118c00ba7cff6d312a/users/39f6e1501362435d8485ac5eb4b9f315/roles/65feabf3b27944a8b01a86c8ac608457` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:34:57.264483 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a89e36e-07cb-401a-b984-5f42ce52e5fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:34:57.265732 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a89e36e-07cb-401a-b984-5f42ce52e5fa admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:34:57.266819 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a89e36e-07cb-401a-b984-5f42ce52e5fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:34:57.267280 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a89e36e-07cb-401a-b984-5f42ce52e5fa admin admin] PATH_INFO: `/v3/projects/e42dc6e1749442ccb96a7a6a588387d8/users/d55ef052e0434ddd85806f6d67c8ce7b/roles/65feabf3b27944a8b01a86c8ac608457` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:34:57.281714 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adfd2d1-90ac-4824-968d-662a2af0f8dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0695b3617d845118c00ba7cff6d312a, user_id=39f6e1501362435d8485ac5eb4b9f315, role_id=65feabf3b27944a8b01a86c8ac608457)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:34:57.284310 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a89e36e-07cb-401a-b984-5f42ce52e5fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=e42dc6e1749442ccb96a7a6a588387d8, user_id=d55ef052e0434ddd85806f6d67c8ce7b, role_id=65feabf3b27944a8b01a86c8ac608457)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:34:57.284567 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adfd2d1-90ac-4824-968d-662a2af0f8dd admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:34:57.292421 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a89e36e-07cb-401a-b984-5f42ce52e5fa admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:34:57.421310 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 818/1611] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:34:57 2022] PUT /identity/v3/projects/d0695b3617d845118c00ba7cff6d312a/users/39f6e1501362435d8485ac5eb4b9f315/roles/65feabf3b27944a8b01a86c8ac608457 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:34:57.422256 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 794/1612] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:34:57 2022] PUT /identity/v3/projects/e42dc6e1749442ccb96a7a6a588387d8/users/d55ef052e0434ddd85806f6d67c8ce7b/roles/65feabf3b27944a8b01a86c8ac608457 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:34:57.423314 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8bbf19-1cdc-4919-b521-bfada6478c7b None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:34:57.423579 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a89e36e-07cb-401a-b984-5f42ce52e5fa admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:34:57.423707 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8bbf19-1cdc-4919-b521-bfada6478c7b None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:34:57.423874 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8bbf19-1cdc-4919-b521-bfada6478c7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:34:57.456099 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-cd8bbf19-1cdc-4919-b521-bfada6478c7b None None] MFA Rules not processed for user `932c4d9cb0034247905e0bc72b6ccbce`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:34:57.457452 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-cd8bbf19-1cdc-4919-b521-bfada6478c7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:34:57.501669 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:34:57.502456 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a89e36e-07cb-401a-b984-5f42ce52e5fa admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:34:57.539035 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 819/1613] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:34:57 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:34:57.540509 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8bbf19-1cdc-4919-b521-bfada6478c7b None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:34:57.577282 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365b308f-e170-41c3-b564-92cf489ddbfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:34:57.578271 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365b308f-e170-41c3-b564-92cf489ddbfe admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:34:57.578454 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365b308f-e170-41c3-b564-92cf489ddbfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:34:57.578784 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365b308f-e170-41c3-b564-92cf489ddbfe admin admin] PATH_INFO: `/v3/projects/1d493928bffa467d8ca39fe0f5403cbf/users/d9c2326a653a49d89f03331a9888ff60/roles/d4aa3324f0c84887b41d0dc4acf7f30e` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:34:57.616002 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365b308f-e170-41c3-b564-92cf489ddbfe admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d493928bffa467d8ca39fe0f5403cbf, user_id=d9c2326a653a49d89f03331a9888ff60, role_id=d4aa3324f0c84887b41d0dc4acf7f30e)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:34:57.620476 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365b308f-e170-41c3-b564-92cf489ddbfe admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:34:57.639803 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8bbf19-1cdc-4919-b521-bfada6478c7b None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:34:57.644019 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84af1591-024e-4209-a29b-20615de5dcbf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eaf19a259c2741cea375a38763e33cdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:34:57.644705 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84af1591-024e-4209-a29b-20615de5dcbf service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:34:57.644995 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84af1591-024e-4209-a29b-20615de5dcbf service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:34:57.645305 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84af1591-024e-4209-a29b-20615de5dcbf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:34:57.654946 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84af1591-024e-4209-a29b-20615de5dcbf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:34:57.656467 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84af1591-024e-4209-a29b-20615de5dcbf service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:34:57.666584 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 820/1614] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:34:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:34:57.667837 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ea65cf-ed8b-43df-9cf8-ae8bcb1e5748 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:34:57.668127 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ea65cf-ed8b-43df-9cf8-ae8bcb1e5748 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:34:57.668406 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ea65cf-ed8b-43df-9cf8-ae8bcb1e5748 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:34:57.690349 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-a8ea65cf-ed8b-43df-9cf8-ae8bcb1e5748 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:34:57.691677 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-a8ea65cf-ed8b-43df-9cf8-ae8bcb1e5748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:34:57.699537 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 821/1615] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:34:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:34:57.700753 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d961799f-52fe-450b-b995-5738e0e54271 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:34:57.701043 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d961799f-52fe-450b-b995-5738e0e54271 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:34:57.701324 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d961799f-52fe-450b-b995-5738e0e54271 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:34:57.719325 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-d961799f-52fe-450b-b995-5738e0e54271 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:34:57.720543 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-d961799f-52fe-450b-b995-5738e0e54271 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:34:57.729951 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 795/1616] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:34:57 2022] PUT /identity/v3/projects/1d493928bffa467d8ca39fe0f5403cbf/users/d9c2326a653a49d89f03331a9888ff60/roles/d4aa3324f0c84887b41d0dc4acf7f30e => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:34:57.731143 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02783e3e-d1eb-470d-9306-cbff87bb7994 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:34:57.731423 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02783e3e-d1eb-470d-9306-cbff87bb7994 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:34:57.731708 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02783e3e-d1eb-470d-9306-cbff87bb7994 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:34:57.773927 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-02783e3e-d1eb-470d-9306-cbff87bb7994 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:34:57.776803 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-02783e3e-d1eb-470d-9306-cbff87bb7994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:34:57.785980 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 822/1617] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:34:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:34:57.787819 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04658e6-4ecb-4244-b2c3-6b0c4cd9713d None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:34:57.788184 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04658e6-4ecb-4244-b2c3-6b0c4cd9713d None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:34:57.788638 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04658e6-4ecb-4244-b2c3-6b0c4cd9713d None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:34:57.792108 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 823/1618] 10.222.0.66 () {54 vars in 962 bytes} [Mon Jul 18 02:34:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:34:57.792108 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04658e6-4ecb-4244-b2c3-6b0c4cd9713d None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:34:57.811248 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 796/1619] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:34:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:34:57.811248 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02783e3e-d1eb-470d-9306-cbff87bb7994 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:34:57.833848 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02783e3e-d1eb-470d-9306-cbff87bb7994 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:34:57.838741 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f244fea9-6a81-43e6-a311-e42d7f503729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:34:57.839488 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f244fea9-6a81-43e6-a311-e42d7f503729 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:34:57.839845 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f244fea9-6a81-43e6-a311-e42d7f503729 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:34:57.840202 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f244fea9-6a81-43e6-a311-e42d7f503729 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:34:57.841057 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f244fea9-6a81-43e6-a311-e42d7f503729 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:34:57.842964 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f244fea9-6a81-43e6-a311-e42d7f503729 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:34:57.855221 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 797/1620] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:34:57 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:34:57.857006 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf93ccb2-34f7-4d07-b6c9-bfc069802f40 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:34:57.857355 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf93ccb2-34f7-4d07-b6c9-bfc069802f40 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:34:57.857698 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf93ccb2-34f7-4d07-b6c9-bfc069802f40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:34:57.863432 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04658e6-4ecb-4244-b2c3-6b0c4cd9713d None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:34:57.867990 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b14de1f-7d26-4c4a-a777-21703bbf3624 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eaf19a259c2741cea375a38763e33cdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:34:57.868709 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b14de1f-7d26-4c4a-a777-21703bbf3624 service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:34:57.869076 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b14de1f-7d26-4c4a-a777-21703bbf3624 service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:34:57.869309 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b14de1f-7d26-4c4a-a777-21703bbf3624 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:34:57.879492 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:34:57.879868 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b14de1f-7d26-4c4a-a777-21703bbf3624 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:34:57.881314 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b14de1f-7d26-4c4a-a777-21703bbf3624 service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:34:57.893101 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 824/1621] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:34:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:34:57.893888 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b14de1f-7d26-4c4a-a777-21703bbf3624 service neutron] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:34:57.903354 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-cf93ccb2-34f7-4d07-b6c9-bfc069802f40 None None] MFA Rules not processed for user `80b138831bc54c7b986d50092d62d4af`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:34:57.904435 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b14de1f-7d26-4c4a-a777-21703bbf3624 service neutron] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:34:57.904587 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-cf93ccb2-34f7-4d07-b6c9-bfc069802f40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:34:57.908905 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a1f146-19a2-4056-bf7f-0d919e545d0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eaf19a259c2741cea375a38763e33cdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:34:57.909931 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a1f146-19a2-4056-bf7f-0d919e545d0f service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:34:57.910221 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a1f146-19a2-4056-bf7f-0d919e545d0f service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:34:57.910649 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a1f146-19a2-4056-bf7f-0d919e545d0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:34:57.922767 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a1f146-19a2-4056-bf7f-0d919e545d0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:34:57.924723 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a1f146-19a2-4056-bf7f-0d919e545d0f service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:34:57.938599 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 825/1622] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:34:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:34:57.939264 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a1f146-19a2-4056-bf7f-0d919e545d0f service neutron] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:34:57.947746 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:34:57.953281 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a1f146-19a2-4056-bf7f-0d919e545d0f service neutron] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:34:57.958158 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687fd130-97db-4058-afae-a3117cc59d08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eaf19a259c2741cea375a38763e33cdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:34:57.958997 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687fd130-97db-4058-afae-a3117cc59d08 service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:34:57.959386 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687fd130-97db-4058-afae-a3117cc59d08 service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:34:57.959767 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687fd130-97db-4058-afae-a3117cc59d08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:34:57.972498 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687fd130-97db-4058-afae-a3117cc59d08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:34:57.974262 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687fd130-97db-4058-afae-a3117cc59d08 service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:34:57.979214 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 798/1623] 10.222.0.66 () {58 vars in 1059 bytes} [Mon Jul 18 02:34:57 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:34:57.981524 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf93ccb2-34f7-4d07-b6c9-bfc069802f40 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:34:57.991353 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf93ccb2-34f7-4d07-b6c9-bfc069802f40 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:34:57.992558 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 826/1624] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:34:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:34:57.993283 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687fd130-97db-4058-afae-a3117cc59d08 service neutron] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:34:57.997092 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49841e7-30b8-48ba-9d36-e3c0338ee141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:34:57.997720 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49841e7-30b8-48ba-9d36-e3c0338ee141 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:34:57.998014 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49841e7-30b8-48ba-9d36-e3c0338ee141 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:34:57.998405 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49841e7-30b8-48ba-9d36-e3c0338ee141 admin admin] PATH_INFO: `/v3/projects/1d493928bffa467d8ca39fe0f5403cbf/users/d9c2326a653a49d89f03331a9888ff60/roles/d4aa3324f0c84887b41d0dc4acf7f30e` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:34:58.001535 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49841e7-30b8-48ba-9d36-e3c0338ee141 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d493928bffa467d8ca39fe0f5403cbf, user_id=d9c2326a653a49d89f03331a9888ff60, role_id=d4aa3324f0c84887b41d0dc4acf7f30e)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:34:58.004445 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49841e7-30b8-48ba-9d36-e3c0338ee141 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:34:58.007015 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687fd130-97db-4058-afae-a3117cc59d08 service neutron] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:34:58.012158 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c2006c-a938-46fe-925a-a2f0261a31ba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80b138831bc54c7b986d50092d62d4af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:34:58.012814 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c2006c-a938-46fe-925a-a2f0261a31ba service nova] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:34:58.013152 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c2006c-a938-46fe-925a-a2f0261a31ba service nova] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:34:58.013433 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c2006c-a938-46fe-925a-a2f0261a31ba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:34:58.022177 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 799/1625] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:34:57 2022] PUT /identity/v3/projects/1d493928bffa467d8ca39fe0f5403cbf/users/d9c2326a653a49d89f03331a9888ff60/roles/d4aa3324f0c84887b41d0dc4acf7f30e => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:34:58.029009 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4960b3bb-1cee-4451-ae23-d8000c0dbe9c None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:34:58.029319 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4960b3bb-1cee-4451-ae23-d8000c0dbe9c None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:34:58.030000 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4960b3bb-1cee-4451-ae23-d8000c0dbe9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:34:58.060377 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-4960b3bb-1cee-4451-ae23-d8000c0dbe9c None None] MFA Rules not processed for user `d9c2326a653a49d89f03331a9888ff60`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:34:58.061610 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-4960b3bb-1cee-4451-ae23-d8000c0dbe9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:34:58.095778 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c2006c-a938-46fe-925a-a2f0261a31ba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:34:58.097516 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c2006c-a938-46fe-925a-a2f0261a31ba service nova] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:34:58.111863 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 827/1626] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:34:57 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:34:58.159338 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 800/1627] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:34:58 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:34:58.177611 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c2006c-a938-46fe-925a-a2f0261a31ba service nova] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:34:58.274646 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c2006c-a938-46fe-925a-a2f0261a31ba service nova] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:34:58.359835 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7b4ab6-8dd5-49fc-ba6c-e3c4d22bb01c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd49388facdf446885e55b45ef91f01e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:34:58.360754 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7b4ab6-8dd5-49fc-ba6c-e3c4d22bb01c service cinder] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:34:58.361159 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7b4ab6-8dd5-49fc-ba6c-e3c4d22bb01c service cinder] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:34:58.361564 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7b4ab6-8dd5-49fc-ba6c-e3c4d22bb01c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:34:58.373023 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7b4ab6-8dd5-49fc-ba6c-e3c4d22bb01c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:34:58.374511 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7b4ab6-8dd5-49fc-ba6c-e3c4d22bb01c service cinder] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:34:58.386115 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 828/1628] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:34:58 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:34:58.526733 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd12ea89-79c4-479f-9f81-9ab7fc09fb05 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:34:58.527213 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd12ea89-79c4-479f-9f81-9ab7fc09fb05 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:34:58.527411 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd12ea89-79c4-479f-9f81-9ab7fc09fb05 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:34:58.528890 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 801/1629] 10.222.0.66 () {54 vars in 912 bytes} [Mon Jul 18 02:34:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:34:58.535197 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037bc457-8f22-4879-bbfa-47aea506cf77 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:34:58.535436 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037bc457-8f22-4879-bbfa-47aea506cf77 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:34:58.535716 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037bc457-8f22-4879-bbfa-47aea506cf77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:34:58.577788 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-037bc457-8f22-4879-bbfa-47aea506cf77 None None] MFA Rules not processed for user `80b138831bc54c7b986d50092d62d4af`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:34:58.579507 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-037bc457-8f22-4879-bbfa-47aea506cf77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:34:58.651878 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:34:58.689680 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 829/1630] 10.222.0.66 () {58 vars in 1009 bytes} [Mon Jul 18 02:34:58 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:34:58.702724 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd12ea89-79c4-479f-9f81-9ab7fc09fb05 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:34:58.715956 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd12ea89-79c4-479f-9f81-9ab7fc09fb05 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:34:58.720736 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3459a6-4a0d-4dae-91ad-d8c94c2fd648 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd49388facdf446885e55b45ef91f01e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:34:58.721467 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3459a6-4a0d-4dae-91ad-d8c94c2fd648 service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:34:58.722039 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3459a6-4a0d-4dae-91ad-d8c94c2fd648 service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:34:58.724370 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3459a6-4a0d-4dae-91ad-d8c94c2fd648 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:34:58.745262 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037bc457-8f22-4879-bbfa-47aea506cf77 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:34:58.770225 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037bc457-8f22-4879-bbfa-47aea506cf77 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:34:58.773440 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3459a6-4a0d-4dae-91ad-d8c94c2fd648 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:34:58.775369 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3459a6-4a0d-4dae-91ad-d8c94c2fd648 service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:34:58.775677 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c07b107-0ba9-4dad-9452-2523026fe4b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd49388facdf446885e55b45ef91f01e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:34:58.776828 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c07b107-0ba9-4dad-9452-2523026fe4b0 service cinder] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:34:58.777112 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c07b107-0ba9-4dad-9452-2523026fe4b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:34:58.777387 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c07b107-0ba9-4dad-9452-2523026fe4b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:34:58.798965 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 802/1631] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:34:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:34:58.892488 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c07b107-0ba9-4dad-9452-2523026fe4b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:34:58.895850 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c07b107-0ba9-4dad-9452-2523026fe4b0 service cinder] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:34:58.921486 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 830/1632] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:34:58 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 02:34:59.899585 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a668d00b-d013-4cb1-a9ed-31322c77ec19 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:34:59.900725 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a668d00b-d013-4cb1-a9ed-31322c77ec19 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:34:59.901135 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a668d00b-d013-4cb1-a9ed-31322c77ec19 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:34:59.903258 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 803/1633] 10.222.0.66 () {54 vars in 962 bytes} [Mon Jul 18 02:34:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:34:59.914662 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c8205a-8035-47dd-9dd7-f11b0d745cb7 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:34:59.915384 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c8205a-8035-47dd-9dd7-f11b0d745cb7 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:34:59.915644 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c8205a-8035-47dd-9dd7-f11b0d745cb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:34:59.946351 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-92c8205a-8035-47dd-9dd7-f11b0d745cb7 None None] MFA Rules not processed for user `80b138831bc54c7b986d50092d62d4af`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:34:59.947756 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-92c8205a-8035-47dd-9dd7-f11b0d745cb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:34:59.959080 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 831/1634] 10.222.0.66 () {58 vars in 1059 bytes} [Mon Jul 18 02:34:59 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:34:59.965316 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc6c650-8019-4fb2-9bdb-0f10256b4cc0 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:34:59.965605 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc6c650-8019-4fb2-9bdb-0f10256b4cc0 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:34:59.965886 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc6c650-8019-4fb2-9bdb-0f10256b4cc0 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:34:59.967319 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 804/1635] 10.222.0.66 () {54 vars in 912 bytes} [Mon Jul 18 02:34:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:34:59.973230 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b40ef24-1c91-45a0-a699-fe1b76b4b637 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:34:59.973610 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b40ef24-1c91-45a0-a699-fe1b76b4b637 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:34:59.973882 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b40ef24-1c91-45a0-a699-fe1b76b4b637 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:34:59.992802 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-0b40ef24-1c91-45a0-a699-fe1b76b4b637 None None] MFA Rules not processed for user `80b138831bc54c7b986d50092d62d4af`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:34:59.994062 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-0b40ef24-1c91-45a0-a699-fe1b76b4b637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:00.003949 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 832/1636] 10.222.0.66 () {58 vars in 1009 bytes} [Mon Jul 18 02:34:59 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:35:00.016925 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc6c650-8019-4fb2-9bdb-0f10256b4cc0 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:00.027725 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc6c650-8019-4fb2-9bdb-0f10256b4cc0 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:00.032523 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5e64fa-982b-49e3-b3a9-4710bd25dda2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd49388facdf446885e55b45ef91f01e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:00.037136 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5e64fa-982b-49e3-b3a9-4710bd25dda2 service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:00.037136 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5e64fa-982b-49e3-b3a9-4710bd25dda2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:00.037136 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5e64fa-982b-49e3-b3a9-4710bd25dda2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:00.044693 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5e64fa-982b-49e3-b3a9-4710bd25dda2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:00.046148 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5e64fa-982b-49e3-b3a9-4710bd25dda2 service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:00.065786 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 805/1637] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:35:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:35:00.576186 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7adef3-28d9-4d7a-98ae-9331ef2b6a6f None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:00.576186 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7adef3-28d9-4d7a-98ae-9331ef2b6a6f None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:00.576186 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7adef3-28d9-4d7a-98ae-9331ef2b6a6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:00.576186 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-6d7adef3-28d9-4d7a-98ae-9331ef2b6a6f None None] MFA Rules not processed for user `d55ef052e0434ddd85806f6d67c8ce7b`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:35:00.576186 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-6d7adef3-28d9-4d7a-98ae-9331ef2b6a6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:00.705034 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 833/1638] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:35:00 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:35:00.707305 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5e64fa-982b-49e3-b3a9-4710bd25dda2 service cinder] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:00.809041 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:35:00.871384 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5e64fa-982b-49e3-b3a9-4710bd25dda2 service cinder] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:00.877183 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0eef28-1610-43b6-b2c4-24ef8d5ba807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:00.887965 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0eef28-1610-43b6-b2c4-24ef8d5ba807 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:00.888271 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0eef28-1610-43b6-b2c4-24ef8d5ba807 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:00.888555 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0eef28-1610-43b6-b2c4-24ef8d5ba807 admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:00.889509 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0eef28-1610-43b6-b2c4-24ef8d5ba807 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:00.892352 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0eef28-1610-43b6-b2c4-24ef8d5ba807 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:01.017811 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7adef3-28d9-4d7a-98ae-9331ef2b6a6f None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:01.020547 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 806/1639] 10.222.0.66 () {60 vars in 1152 bytes} [Mon Jul 18 02:35:00 2022] POST /identity/v3/projects => generated 401 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:35:01.035431 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7adef3-28d9-4d7a-98ae-9331ef2b6a6f None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:01.129353 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:35:01.155099 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9d6abb-57ff-4961-9864-833aaf86ab8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:01.156514 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9d6abb-57ff-4961-9864-833aaf86ab8b admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:01.157398 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9d6abb-57ff-4961-9864-833aaf86ab8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:01.157747 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9d6abb-57ff-4961-9864-833aaf86ab8b admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:01.158677 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9d6abb-57ff-4961-9864-833aaf86ab8b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:01.160982 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9d6abb-57ff-4961-9864-833aaf86ab8b admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:01.257924 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 834/1640] 10.222.0.66 () {60 vars in 1143 bytes} [Mon Jul 18 02:35:01 2022] POST /identity/v3/users => generated 307 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:35:01.259646 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0eef28-1610-43b6-b2c4-24ef8d5ba807 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:01.279155 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0eef28-1610-43b6-b2c4-24ef8d5ba807 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:01.308694 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b3d02a-29b6-4c26-b1a8-9b97acd00b90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:01.308694 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b3d02a-29b6-4c26-b1a8-9b97acd00b90 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:01.308694 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b3d02a-29b6-4c26-b1a8-9b97acd00b90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:01.308694 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b3d02a-29b6-4c26-b1a8-9b97acd00b90 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:01.308978 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b3d02a-29b6-4c26-b1a8-9b97acd00b90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:01.312795 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b3d02a-29b6-4c26-b1a8-9b97acd00b90 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:01.349045 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 807/1641] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:35:01 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:35:01.351585 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9d6abb-57ff-4961-9864-833aaf86ab8b admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:01.368744 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9d6abb-57ff-4961-9864-833aaf86ab8b admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:01.380244 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0a8982-f52c-4846-bf88-917a99672ecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:01.381613 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0a8982-f52c-4846-bf88-917a99672ecc admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:01.382157 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0a8982-f52c-4846-bf88-917a99672ecc admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:01.382594 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0a8982-f52c-4846-bf88-917a99672ecc admin admin] PATH_INFO: `/v3/projects/d833251e6cee4ce396d02f001221c99b/users/050826b30ece4dda86e5049bfd6c6d01/roles/d4aa3324f0c84887b41d0dc4acf7f30e` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:01.421960 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0a8982-f52c-4846-bf88-917a99672ecc admin admin] RBAC: Authorizing `identity:create_grant(project_id=d833251e6cee4ce396d02f001221c99b, user_id=050826b30ece4dda86e5049bfd6c6d01, role_id=d4aa3324f0c84887b41d0dc4acf7f30e)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:01.426900 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0a8982-f52c-4846-bf88-917a99672ecc admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:01.551575 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 835/1642] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:35:01 2022] PUT /identity/v3/projects/d833251e6cee4ce396d02f001221c99b/users/050826b30ece4dda86e5049bfd6c6d01/roles/d4aa3324f0c84887b41d0dc4acf7f30e => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:35:01.555548 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b3d02a-29b6-4c26-b1a8-9b97acd00b90 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:01.603082 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b3d02a-29b6-4c26-b1a8-9b97acd00b90 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:01.743970 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ddbbdc-9f05-4294-96a6-5d5a6333d46d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:01.744874 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ddbbdc-9f05-4294-96a6-5d5a6333d46d admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:01.745333 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ddbbdc-9f05-4294-96a6-5d5a6333d46d admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:01.745749 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ddbbdc-9f05-4294-96a6-5d5a6333d46d admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:01.746655 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ddbbdc-9f05-4294-96a6-5d5a6333d46d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:01.750229 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ddbbdc-9f05-4294-96a6-5d5a6333d46d admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:01.781501 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 808/1643] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:35:01 2022] GET /identity/v3/roles => generated 1420 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:35:01.785941 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0a8982-f52c-4846-bf88-917a99672ecc admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:01.823240 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0a8982-f52c-4846-bf88-917a99672ecc admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:01.841270 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f164227-6ac8-4f04-9374-14c77613f1fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:01.843273 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f164227-6ac8-4f04-9374-14c77613f1fa admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:01.843736 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f164227-6ac8-4f04-9374-14c77613f1fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:01.844374 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f164227-6ac8-4f04-9374-14c77613f1fa admin admin] PATH_INFO: `/v3/projects/d833251e6cee4ce396d02f001221c99b/users/050826b30ece4dda86e5049bfd6c6d01/roles/d4aa3324f0c84887b41d0dc4acf7f30e` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:01.850251 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f164227-6ac8-4f04-9374-14c77613f1fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=d833251e6cee4ce396d02f001221c99b, user_id=050826b30ece4dda86e5049bfd6c6d01, role_id=d4aa3324f0c84887b41d0dc4acf7f30e)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:01.859419 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f164227-6ac8-4f04-9374-14c77613f1fa admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:01.889951 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 836/1644] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:35:01 2022] PUT /identity/v3/projects/d833251e6cee4ce396d02f001221c99b/users/050826b30ece4dda86e5049bfd6c6d01/roles/d4aa3324f0c84887b41d0dc4acf7f30e => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 02:35:06.187336 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abac96cc-fb38-47de-8e48-7b50c4b12219 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:06.187336 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abac96cc-fb38-47de-8e48-7b50c4b12219 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:06.188121 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abac96cc-fb38-47de-8e48-7b50c4b12219 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:06.255137 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-abac96cc-fb38-47de-8e48-7b50c4b12219 None None] MFA Rules not processed for user `050826b30ece4dda86e5049bfd6c6d01`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:35:06.256319 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-abac96cc-fb38-47de-8e48-7b50c4b12219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:06.438124 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 809/1645] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:35:06 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:35:10.094240 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbffdeb3-f368-415b-8f6c-f1db7867103d None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:10.094240 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbffdeb3-f368-415b-8f6c-f1db7867103d None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:10.094240 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbffdeb3-f368-415b-8f6c-f1db7867103d None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:10.096460 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 837/1646] 10.222.0.66 () {54 vars in 921 bytes} [Mon Jul 18 02:35:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:35:10.172533 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbac9d5-e97d-4c59-80af-16dbb7290c16 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:10.172533 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbac9d5-e97d-4c59-80af-16dbb7290c16 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:10.172533 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbac9d5-e97d-4c59-80af-16dbb7290c16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:10.214844 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-fdbac9d5-e97d-4c59-80af-16dbb7290c16 None None] MFA Rules not processed for user `88936b31709b429f8b6777c673ed7b8a`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:35:10.216884 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-fdbac9d5-e97d-4c59-80af-16dbb7290c16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:10.272336 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:35:10.358793 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 810/1647] 10.222.0.66 () {58 vars in 1018 bytes} [Mon Jul 18 02:35:10 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:35:10.553702 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4b5440-0baa-4538-a16e-3cc5889d39af None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:10.554492 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4b5440-0baa-4538-a16e-3cc5889d39af None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:10.555271 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4b5440-0baa-4538-a16e-3cc5889d39af None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:10.557867 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 838/1648] 10.222.0.66 () {54 vars in 961 bytes} [Mon Jul 18 02:35:10 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:35:10.564871 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e96e167-0cab-4f98-b10f-8ed1e6d74b4a None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:10.565168 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e96e167-0cab-4f98-b10f-8ed1e6d74b4a None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:10.565449 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e96e167-0cab-4f98-b10f-8ed1e6d74b4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:10.604932 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-9e96e167-0cab-4f98-b10f-8ed1e6d74b4a None None] MFA Rules not processed for user `88936b31709b429f8b6777c673ed7b8a`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:35:10.605886 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-9e96e167-0cab-4f98-b10f-8ed1e6d74b4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:10.643511 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 811/1649] 10.222.0.66 () {58 vars in 1058 bytes} [Mon Jul 18 02:35:10 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:35:10.653000 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4b5440-0baa-4538-a16e-3cc5889d39af None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:10.685420 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4b5440-0baa-4538-a16e-3cc5889d39af None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:10.695268 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dfcad8-a2c2-4471-b203-68f5f642ed23 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88936b31709b429f8b6777c673ed7b8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:10.695948 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dfcad8-a2c2-4471-b203-68f5f642ed23 service placement] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:10.696225 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dfcad8-a2c2-4471-b203-68f5f642ed23 service placement] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:10.698397 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dfcad8-a2c2-4471-b203-68f5f642ed23 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:10.723066 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dfcad8-a2c2-4471-b203-68f5f642ed23 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:10.724610 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dfcad8-a2c2-4471-b203-68f5f642ed23 service placement] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:10.763071 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 839/1650] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:35:10 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:35:11.103419 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0859080a-eafb-4870-bf94-c5eebf4977d6 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:11.105546 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0859080a-eafb-4870-bf94-c5eebf4977d6 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:11.106033 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0859080a-eafb-4870-bf94-c5eebf4977d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:11.190499 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-0859080a-eafb-4870-bf94-c5eebf4977d6 None None] MFA Rules not processed for user `879f7a4cf6664e76bddffc804658b250`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:35:11.202717 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-0859080a-eafb-4870-bf94-c5eebf4977d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:11.437576 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 812/1651] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:35:11 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 335 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:35:11.446073 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dfcad8-a2c2-4471-b203-68f5f642ed23 service placement] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:11.538035 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab97dbf-1a97-46b1-94e8-c0f25bf76ceb None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:11.539318 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab97dbf-1a97-46b1-94e8-c0f25bf76ceb None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:11.539318 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab97dbf-1a97-46b1-94e8-c0f25bf76ceb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:11.620551 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-5ab97dbf-1a97-46b1-94e8-c0f25bf76ceb None None] MFA Rules not processed for user `39f6e1501362435d8485ac5eb4b9f315`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:35:11.620551 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-5ab97dbf-1a97-46b1-94e8-c0f25bf76ceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:11.785668 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dfcad8-a2c2-4471-b203-68f5f642ed23 service placement] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:11.816673 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed73168-4c54-441f-a146-b3c4bcc25803 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80b138831bc54c7b986d50092d62d4af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:11.817850 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed73168-4c54-441f-a146-b3c4bcc25803 service nova] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:11.818400 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed73168-4c54-441f-a146-b3c4bcc25803 service nova] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:11.819050 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed73168-4c54-441f-a146-b3c4bcc25803 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:11.852957 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed73168-4c54-441f-a146-b3c4bcc25803 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:11.856907 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed73168-4c54-441f-a146-b3c4bcc25803 service nova] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:11.896479 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 840/1652] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:35:11 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:35:11.898412 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed73168-4c54-441f-a146-b3c4bcc25803 service nova] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:11.921151 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 813/1653] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:35:11 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 383 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:35:11.926379 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc0d762-72a1-418a-93a8-9c258c59e394 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:11.926809 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc0d762-72a1-418a-93a8-9c258c59e394 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:11.927059 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc0d762-72a1-418a-93a8-9c258c59e394 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:12.001568 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-6bc0d762-72a1-418a-93a8-9c258c59e394 None None] MFA Rules not processed for user `39f6e1501362435d8485ac5eb4b9f315`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:35:12.009831 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-6bc0d762-72a1-418a-93a8-9c258c59e394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:12.034806 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 814/1654] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:35:11 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:35:12.056121 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc0d762-72a1-418a-93a8-9c258c59e394 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:12.094726 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc0d762-72a1-418a-93a8-9c258c59e394 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:12.102456 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed73168-4c54-441f-a146-b3c4bcc25803 service nova] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:12.115505 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4302c5ba-bb11-4f5e-877a-aa9da1ade719 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80b138831bc54c7b986d50092d62d4af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:12.116185 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4302c5ba-bb11-4f5e-877a-aa9da1ade719 service nova] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:12.116471 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4302c5ba-bb11-4f5e-877a-aa9da1ade719 service nova] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:12.122116 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4302c5ba-bb11-4f5e-877a-aa9da1ade719 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:12.124983 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183a374d-7574-4a5d-afca-955f69408bec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:12.126410 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183a374d-7574-4a5d-afca-955f69408bec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:12.126717 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183a374d-7574-4a5d-afca-955f69408bec admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:12.126956 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183a374d-7574-4a5d-afca-955f69408bec admin admin] PATH_INFO: `/v3/users/932c4d9cb0034247905e0bc72b6ccbce` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:12.134428 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183a374d-7574-4a5d-afca-955f69408bec admin admin] RBAC: Authorizing `identity:delete_user(user_id=932c4d9cb0034247905e0bc72b6ccbce)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:12.140514 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183a374d-7574-4a5d-afca-955f69408bec admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:12.150592 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4302c5ba-bb11-4f5e-877a-aa9da1ade719 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:12.158434 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4302c5ba-bb11-4f5e-877a-aa9da1ade719 service nova] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:12.204485 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 815/1655] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:35:12 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 02:35:12.235044 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:35:12.490909 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-183a374d-7574-4a5d-afca-955f69408bec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '932c4d9cb0034247905e0bc72b6ccbce'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:12.591506 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-183a374d-7574-4a5d-afca-955f69408bec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '932c4d9cb0034247905e0bc72b6ccbce'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:12.605214 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-183a374d-7574-4a5d-afca-955f69408bec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '932c4d9cb0034247905e0bc72b6ccbce'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:12.606545 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-183a374d-7574-4a5d-afca-955f69408bec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '932c4d9cb0034247905e0bc72b6ccbce'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:12.645209 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-183a374d-7574-4a5d-afca-955f69408bec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '932c4d9cb0034247905e0bc72b6ccbce'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:12.657923 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 841/1656] 10.222.0.66 () {58 vars in 1223 bytes} [Mon Jul 18 02:35:11 2022] DELETE /identity/v3/users/932c4d9cb0034247905e0bc72b6ccbce => generated 0 bytes in 757 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:35:12.659649 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1158d4f-f13e-4086-a32d-fd2ae1f827a3 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:12.660042 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1158d4f-f13e-4086-a32d-fd2ae1f827a3 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:12.664064 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1158d4f-f13e-4086-a32d-fd2ae1f827a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:12.714882 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-b1158d4f-f13e-4086-a32d-fd2ae1f827a3 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:35:12.719676 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-b1158d4f-f13e-4086-a32d-fd2ae1f827a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:12.777498 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:35:12.921519 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 816/1657] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:35:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:35:12.957375 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183a374d-7574-4a5d-afca-955f69408bec admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:12.959141 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-183a374d-7574-4a5d-afca-955f69408bec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:13.109760 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183a374d-7574-4a5d-afca-955f69408bec admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:13.121668 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0999dd-7d7e-4d38-a411-06a3840de619 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eaf19a259c2741cea375a38763e33cdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:13.122396 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0999dd-7d7e-4d38-a411-06a3840de619 service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:13.122690 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0999dd-7d7e-4d38-a411-06a3840de619 service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:13.123196 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0999dd-7d7e-4d38-a411-06a3840de619 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:13.139979 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0999dd-7d7e-4d38-a411-06a3840de619 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:13.141806 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0999dd-7d7e-4d38-a411-06a3840de619 service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:13.175241 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 842/1658] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:35:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:35:14.042983 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b221a9ec-af28-427d-a34d-3dbb4d714bd3 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:14.043448 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b221a9ec-af28-427d-a34d-3dbb4d714bd3 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:14.043536 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b221a9ec-af28-427d-a34d-3dbb4d714bd3 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:14.045313 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 817/1659] 10.222.0.66 () {54 vars in 912 bytes} [Mon Jul 18 02:35:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:35:14.051883 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf5cc1c-2622-477a-a4ac-ba8cd0fa215d None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:14.052575 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf5cc1c-2622-477a-a4ac-ba8cd0fa215d None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:14.053462 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf5cc1c-2622-477a-a4ac-ba8cd0fa215d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:14.074096 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be172b4-004e-490b-a0fa-f804adc44375 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:14.074396 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be172b4-004e-490b-a0fa-f804adc44375 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:14.074678 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be172b4-004e-490b-a0fa-f804adc44375 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:14.076078 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 818/1660] 10.222.0.66 () {54 vars in 921 bytes} [Mon Jul 18 02:35:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 02:35:14.100972 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cac65d-7020-43fe-9603-c7f33e8fc88c None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:14.101847 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cac65d-7020-43fe-9603-c7f33e8fc88c None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:14.102352 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cac65d-7020-43fe-9603-c7f33e8fc88c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:14.132828 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-30cac65d-7020-43fe-9603-c7f33e8fc88c None None] MFA Rules not processed for user `88936b31709b429f8b6777c673ed7b8a`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:35:14.135510 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-30cac65d-7020-43fe-9603-c7f33e8fc88c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:14.205390 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-5bf5cc1c-2622-477a-a4ac-ba8cd0fa215d None None] MFA Rules not processed for user `eaf19a259c2741cea375a38763e33cdc`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:35:14.209862 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-5bf5cc1c-2622-477a-a4ac-ba8cd0fa215d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:14.246454 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 843/1661] 10.222.0.66 () {58 vars in 1009 bytes} [Mon Jul 18 02:35:14 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:35:14.287888 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf5cc1c-2622-477a-a4ac-ba8cd0fa215d None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:14.291249 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-5bf5cc1c-2622-477a-a4ac-ba8cd0fa215d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:14.301571 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 819/1662] 10.222.0.66 () {58 vars in 1018 bytes} [Mon Jul 18 02:35:14 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:35:14.315942 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cac65d-7020-43fe-9603-c7f33e8fc88c None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:14.317832 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-30cac65d-7020-43fe-9603-c7f33e8fc88c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:14.317947 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf5cc1c-2622-477a-a4ac-ba8cd0fa215d None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:14.331721 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff59de3-7686-4c58-a312-f1d817532a9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eaf19a259c2741cea375a38763e33cdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:14.332400 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff59de3-7686-4c58-a312-f1d817532a9d service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:14.332689 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff59de3-7686-4c58-a312-f1d817532a9d service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:14.332965 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff59de3-7686-4c58-a312-f1d817532a9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:14.352092 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff59de3-7686-4c58-a312-f1d817532a9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:14.354625 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff59de3-7686-4c58-a312-f1d817532a9d service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:14.356288 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cac65d-7020-43fe-9603-c7f33e8fc88c None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:14.368038 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 844/1663] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:35:14 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:35:14.377234 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fb54bd-1ae9-4700-af17-f5e3425bdcfa service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88936b31709b429f8b6777c673ed7b8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:14.378119 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fb54bd-1ae9-4700-af17-f5e3425bdcfa service placement] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:14.378421 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fb54bd-1ae9-4700-af17-f5e3425bdcfa service placement] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:14.378553 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fb54bd-1ae9-4700-af17-f5e3425bdcfa service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:14.402673 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fb54bd-1ae9-4700-af17-f5e3425bdcfa service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:14.404318 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fb54bd-1ae9-4700-af17-f5e3425bdcfa service placement] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:14.446689 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 820/1664] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:35:14 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 02:35:14.461363 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff59de3-7686-4c58-a312-f1d817532a9d service neutron] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:14.462859 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-cff59de3-7686-4c58-a312-f1d817532a9d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:14.493934 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff59de3-7686-4c58-a312-f1d817532a9d service neutron] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:14.502555 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ab6fba-2e8c-4bcb-b9ed-29b2d4a03df1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:14.503676 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ab6fba-2e8c-4bcb-b9ed-29b2d4a03df1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:14.504129 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ab6fba-2e8c-4bcb-b9ed-29b2d4a03df1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:14.504491 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ab6fba-2e8c-4bcb-b9ed-29b2d4a03df1 admin admin] PATH_INFO: `/v3/users/d55ef052e0434ddd85806f6d67c8ce7b` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:14.506214 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ab6fba-2e8c-4bcb-b9ed-29b2d4a03df1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d55ef052e0434ddd85806f6d67c8ce7b)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:14.508747 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ab6fba-2e8c-4bcb-b9ed-29b2d4a03df1 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:14.727863 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fb54bd-1ae9-4700-af17-f5e3425bdcfa service placement] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:14.730265 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-d3fb54bd-1ae9-4700-af17-f5e3425bdcfa service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:14.765252 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fb54bd-1ae9-4700-af17-f5e3425bdcfa service placement] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:14.788534 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a054970-610a-4269-b620-fccf34a9021c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:14.788534 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a054970-610a-4269-b620-fccf34a9021c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:14.789880 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a054970-610a-4269-b620-fccf34a9021c admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:14.790206 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a054970-610a-4269-b620-fccf34a9021c admin admin] PATH_INFO: `/v3/users/e0bb4bfb06ec4eae87355c6e84328c26` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:14.794606 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a054970-610a-4269-b620-fccf34a9021c admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0bb4bfb06ec4eae87355c6e84328c26)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:14.797933 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: WARNING py.warnings [None req-6a054970-610a-4269-b620-fccf34a9021c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 02:35:14.797933 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: warnings.warn(msg) Jul 18 02:35:14.797933 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: Jul 18 02:35:14.803810 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a054970-610a-4269-b620-fccf34a9021c admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:14.804806 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-c5ab6fba-2e8c-4bcb-b9ed-29b2d4a03df1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd55ef052e0434ddd85806f6d67c8ce7b'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:15.123367 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-c5ab6fba-2e8c-4bcb-b9ed-29b2d4a03df1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd55ef052e0434ddd85806f6d67c8ce7b'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:15.127296 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:35:15.134735 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-c5ab6fba-2e8c-4bcb-b9ed-29b2d4a03df1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd55ef052e0434ddd85806f6d67c8ce7b'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:15.141700 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-c5ab6fba-2e8c-4bcb-b9ed-29b2d4a03df1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd55ef052e0434ddd85806f6d67c8ce7b'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:15.168926 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-c5ab6fba-2e8c-4bcb-b9ed-29b2d4a03df1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd55ef052e0434ddd85806f6d67c8ce7b'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:15.191005 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 845/1665] 10.222.0.66 () {58 vars in 1223 bytes} [Mon Jul 18 02:35:14 2022] DELETE /identity/v3/users/d55ef052e0434ddd85806f6d67c8ce7b => generated 0 bytes in 727 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:35:15.194509 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ab6fba-2e8c-4bcb-b9ed-29b2d4a03df1 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:15.198180 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-c5ab6fba-2e8c-4bcb-b9ed-29b2d4a03df1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:15.322181 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-6a054970-610a-4269-b620-fccf34a9021c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0bb4bfb06ec4eae87355c6e84328c26'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:15.392576 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-6a054970-610a-4269-b620-fccf34a9021c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0bb4bfb06ec4eae87355c6e84328c26'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:15.393450 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-6a054970-610a-4269-b620-fccf34a9021c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0bb4bfb06ec4eae87355c6e84328c26'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:15.422609 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ab6fba-2e8c-4bcb-b9ed-29b2d4a03df1 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:15.445661 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3951c6d4-0cb5-4094-adfd-82f386096dad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:15.446606 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3951c6d4-0cb5-4094-adfd-82f386096dad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:15.446893 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3951c6d4-0cb5-4094-adfd-82f386096dad admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:15.447172 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3951c6d4-0cb5-4094-adfd-82f386096dad admin admin] PATH_INFO: `/v3/projects/5a76c0d5639d42c9bb2387d75f4b0f95` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:15.454729 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-6a054970-610a-4269-b620-fccf34a9021c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0bb4bfb06ec4eae87355c6e84328c26'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:15.458282 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-6a054970-610a-4269-b620-fccf34a9021c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0bb4bfb06ec4eae87355c6e84328c26'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:15.460182 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3951c6d4-0cb5-4094-adfd-82f386096dad admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a76c0d5639d42c9bb2387d75f4b0f95)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:15.461403 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: WARNING py.warnings [None req-3951c6d4-0cb5-4094-adfd-82f386096dad admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 02:35:15.461403 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: warnings.warn(msg) Jul 18 02:35:15.461403 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: Jul 18 02:35:15.462702 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3951c6d4-0cb5-4094-adfd-82f386096dad admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:15.499618 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 821/1666] 10.222.0.66 () {58 vars in 1223 bytes} [Mon Jul 18 02:35:14 2022] DELETE /identity/v3/users/e0bb4bfb06ec4eae87355c6e84328c26 => generated 0 bytes in 772 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:35:15.505348 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636e855f-968f-4c7f-81b5-57127f0872b4 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:15.505913 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636e855f-968f-4c7f-81b5-57127f0872b4 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:15.506225 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636e855f-968f-4c7f-81b5-57127f0872b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:15.536296 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-636e855f-968f-4c7f-81b5-57127f0872b4 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:35:15.537701 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-636e855f-968f-4c7f-81b5-57127f0872b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:15.555468 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:35:15.639427 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 822/1667] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:35:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:35:15.696190 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636e855f-968f-4c7f-81b5-57127f0872b4 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:15.700046 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-636e855f-968f-4c7f-81b5-57127f0872b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:15.877892 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-3951c6d4-0cb5-4094-adfd-82f386096dad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a76c0d5639d42c9bb2387d75f4b0f95', 'request_id': 'req-3951c6d4-0cb5-4094-adfd-82f386096dad'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:15.886682 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636e855f-968f-4c7f-81b5-57127f0872b4 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:15.939111 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-3951c6d4-0cb5-4094-adfd-82f386096dad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a76c0d5639d42c9bb2387d75f4b0f95', 'request_id': 'req-3951c6d4-0cb5-4094-adfd-82f386096dad'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:16.081165 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-3951c6d4-0cb5-4094-adfd-82f386096dad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a76c0d5639d42c9bb2387d75f4b0f95', 'request_id': 'req-3951c6d4-0cb5-4094-adfd-82f386096dad'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:16.097164 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4daf77-60cc-4d27-b453-76cb2a0cf4eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eaf19a259c2741cea375a38763e33cdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:16.098282 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4daf77-60cc-4d27-b453-76cb2a0cf4eb service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:16.098823 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4daf77-60cc-4d27-b453-76cb2a0cf4eb service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:16.099317 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4daf77-60cc-4d27-b453-76cb2a0cf4eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:16.143365 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-3951c6d4-0cb5-4094-adfd-82f386096dad admin admin] The token cache is being invalidate because project 5a76c0d5639d42c9bb2387d75f4b0f95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 02:35:16.143915 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-3951c6d4-0cb5-4094-adfd-82f386096dad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:16.148387 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-3951c6d4-0cb5-4094-adfd-82f386096dad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:16.151871 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 846/1668] 10.222.0.66 () {58 vars in 1232 bytes} [Mon Jul 18 02:35:15 2022] DELETE /identity/v3/projects/5a76c0d5639d42c9bb2387d75f4b0f95 => generated 0 bytes in 962 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:35:16.158327 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38be84be-b9ad-45f8-adda-45256dfba703 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:16.158848 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38be84be-b9ad-45f8-adda-45256dfba703 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:16.159309 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38be84be-b9ad-45f8-adda-45256dfba703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:16.252093 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-38be84be-b9ad-45f8-adda-45256dfba703 None None] MFA Rules not processed for user `94c16391d39f4498a55c89bf2838788f`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:35:16.254415 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-38be84be-b9ad-45f8-adda-45256dfba703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:16.321702 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4daf77-60cc-4d27-b453-76cb2a0cf4eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:16.327783 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4daf77-60cc-4d27-b453-76cb2a0cf4eb service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:16.353017 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 823/1669] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:35:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 657 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:35:16.358910 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c30ccc-d58d-4fb9-9bb5-3cba913dd39b None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:16.359614 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c30ccc-d58d-4fb9-9bb5-3cba913dd39b None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:16.359614 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c30ccc-d58d-4fb9-9bb5-3cba913dd39b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:16.400546 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 847/1670] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:35:16 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:35:16.402972 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b078cb-e891-4d5e-968a-721619e971ce None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:16.403490 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b078cb-e891-4d5e-968a-721619e971ce None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:16.403928 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b078cb-e891-4d5e-968a-721619e971ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:16.407470 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-d5c30ccc-d58d-4fb9-9bb5-3cba913dd39b None None] MFA Rules not processed for user `7ddb197852db4b20a84aa6c0862468ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:35:16.409192 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-d5c30ccc-d58d-4fb9-9bb5-3cba913dd39b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:16.464495 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-02b078cb-e891-4d5e-968a-721619e971ce None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:35:16.468184 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-02b078cb-e891-4d5e-968a-721619e971ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:16.494714 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 848/1671] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:35:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:35:16.495604 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b078cb-e891-4d5e-968a-721619e971ce None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:16.505783 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-02b078cb-e891-4d5e-968a-721619e971ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:16.536104 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 824/1672] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:35:16 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:35:16.537287 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c30ccc-d58d-4fb9-9bb5-3cba913dd39b None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:16.538669 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-d5c30ccc-d58d-4fb9-9bb5-3cba913dd39b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:16.544417 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b078cb-e891-4d5e-968a-721619e971ce None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:16.561357 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0a9f81-f24e-428b-ba65-8255965ac2b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:16.562380 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0a9f81-f24e-428b-ba65-8255965ac2b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:16.562664 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0a9f81-f24e-428b-ba65-8255965ac2b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:16.563023 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0a9f81-f24e-428b-ba65-8255965ac2b2 admin admin] PATH_INFO: `/v3/projects/e42dc6e1749442ccb96a7a6a588387d8` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:16.564302 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0a9f81-f24e-428b-ba65-8255965ac2b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e42dc6e1749442ccb96a7a6a588387d8)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:16.566268 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0a9f81-f24e-428b-ba65-8255965ac2b2 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:16.687980 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c30ccc-d58d-4fb9-9bb5-3cba913dd39b None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:16.699606 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7ab263-0685-42a0-8a88-b5f5549066f1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80b138831bc54c7b986d50092d62d4af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:16.710618 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7ab263-0685-42a0-8a88-b5f5549066f1 service nova] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:16.711263 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7ab263-0685-42a0-8a88-b5f5549066f1 service nova] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:16.716690 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7ab263-0685-42a0-8a88-b5f5549066f1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:16.724794 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7ab263-0685-42a0-8a88-b5f5549066f1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:16.727805 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7ab263-0685-42a0-8a88-b5f5549066f1 service nova] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:16.737488 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-1d0a9f81-f24e-428b-ba65-8255965ac2b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e42dc6e1749442ccb96a7a6a588387d8', 'request_id': 'req-1d0a9f81-f24e-428b-ba65-8255965ac2b2'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:16.746357 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 825/1673] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:35:16 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:35:16.747690 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7ab263-0685-42a0-8a88-b5f5549066f1 service nova] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:16.749822 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-1d0a9f81-f24e-428b-ba65-8255965ac2b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e42dc6e1749442ccb96a7a6a588387d8', 'request_id': 'req-1d0a9f81-f24e-428b-ba65-8255965ac2b2'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:16.750599 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-4c7ab263-0685-42a0-8a88-b5f5549066f1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:16.874129 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7ab263-0685-42a0-8a88-b5f5549066f1 service nova] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:16.876615 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-1d0a9f81-f24e-428b-ba65-8255965ac2b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e42dc6e1749442ccb96a7a6a588387d8', 'request_id': 'req-1d0a9f81-f24e-428b-ba65-8255965ac2b2'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:16.896295 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-1d0a9f81-f24e-428b-ba65-8255965ac2b2 admin admin] The token cache is being invalidate because project e42dc6e1749442ccb96a7a6a588387d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 02:35:16.897130 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-1d0a9f81-f24e-428b-ba65-8255965ac2b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:16.898340 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-1d0a9f81-f24e-428b-ba65-8255965ac2b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:16.901610 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 849/1674] 10.222.0.66 () {58 vars in 1232 bytes} [Mon Jul 18 02:35:16 2022] DELETE /identity/v3/projects/e42dc6e1749442ccb96a7a6a588387d8 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:35:16.905676 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bc4949-fdcf-4dac-81f9-9d3dd713fe26 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:16.908106 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bc4949-fdcf-4dac-81f9-9d3dd713fe26 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:16.908569 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bc4949-fdcf-4dac-81f9-9d3dd713fe26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:16.948403 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-16bc4949-fdcf-4dac-81f9-9d3dd713fe26 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:35:16.952337 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-16bc4949-fdcf-4dac-81f9-9d3dd713fe26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:16.969647 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fc7737-8bcf-4227-955d-18e1aa740662 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88936b31709b429f8b6777c673ed7b8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:16.970542 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fc7737-8bcf-4227-955d-18e1aa740662 service placement] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:16.970880 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fc7737-8bcf-4227-955d-18e1aa740662 service placement] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:16.971161 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fc7737-8bcf-4227-955d-18e1aa740662 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:16.974646 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-16fc7737-8bcf-4227-955d-18e1aa740662 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:16.993523 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fc7737-8bcf-4227-955d-18e1aa740662 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:16.995268 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fc7737-8bcf-4227-955d-18e1aa740662 service placement] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:17.011226 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 826/1675] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:35:16 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:35:17.012278 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fc7737-8bcf-4227-955d-18e1aa740662 service placement] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:17.014517 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-16fc7737-8bcf-4227-955d-18e1aa740662 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:17.106402 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 850/1676] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:35:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:35:17.110207 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bc4949-fdcf-4dac-81f9-9d3dd713fe26 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:17.114503 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-16bc4949-fdcf-4dac-81f9-9d3dd713fe26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:17.147998 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bc4949-fdcf-4dac-81f9-9d3dd713fe26 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:17.156918 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fc7737-8bcf-4227-955d-18e1aa740662 service placement] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:17.160162 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7185332-8a5b-4c24-b1f9-022ce3819479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:17.161121 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7185332-8a5b-4c24-b1f9-022ce3819479 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:17.161412 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7185332-8a5b-4c24-b1f9-022ce3819479 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:17.161694 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7185332-8a5b-4c24-b1f9-022ce3819479 admin admin] PATH_INFO: `/v3/users/050826b30ece4dda86e5049bfd6c6d01` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:17.163474 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7185332-8a5b-4c24-b1f9-022ce3819479 admin admin] RBAC: Authorizing `identity:delete_user(user_id=050826b30ece4dda86e5049bfd6c6d01)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:17.168102 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c932e9f7-d88e-4f18-adb5-6b9d89db1b51 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80b138831bc54c7b986d50092d62d4af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:17.169358 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c932e9f7-d88e-4f18-adb5-6b9d89db1b51 service nova] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:17.172050 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c932e9f7-d88e-4f18-adb5-6b9d89db1b51 service nova] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:17.172495 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c932e9f7-d88e-4f18-adb5-6b9d89db1b51 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:17.173316 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7185332-8a5b-4c24-b1f9-022ce3819479 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:17.174867 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-c932e9f7-d88e-4f18-adb5-6b9d89db1b51 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:17.283646 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:35:17.323419 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c932e9f7-d88e-4f18-adb5-6b9d89db1b51 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:17.325171 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c932e9f7-d88e-4f18-adb5-6b9d89db1b51 service nova] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:17.337525 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 827/1677] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:35:17 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:35:17.338549 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c932e9f7-d88e-4f18-adb5-6b9d89db1b51 service nova] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:17.350909 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c932e9f7-d88e-4f18-adb5-6b9d89db1b51 service nova] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:17.357693 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d54804b-ee4d-4a85-921c-3345638e490c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:17.360900 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d54804b-ee4d-4a85-921c-3345638e490c admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:17.360900 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d54804b-ee4d-4a85-921c-3345638e490c admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:17.360900 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d54804b-ee4d-4a85-921c-3345638e490c admin admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:17.360900 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d54804b-ee4d-4a85-921c-3345638e490c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:35:17.360900 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d54804b-ee4d-4a85-921c-3345638e490c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:17.363070 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d54804b-ee4d-4a85-921c-3345638e490c admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:17.393814 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 828/1678] 10.222.0.66 () {58 vars in 1152 bytes} [Mon Jul 18 02:35:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:35:17.398124 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d54804b-ee4d-4a85-921c-3345638e490c admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:17.424788 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d54804b-ee4d-4a85-921c-3345638e490c admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:17.432009 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-a7185332-8a5b-4c24-b1f9-022ce3819479 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '050826b30ece4dda86e5049bfd6c6d01'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:17.440520 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33078e03-5049-4daa-8822-b8cb8bea5266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:17.441768 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33078e03-5049-4daa-8822-b8cb8bea5266 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:17.444867 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33078e03-5049-4daa-8822-b8cb8bea5266 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:17.446010 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33078e03-5049-4daa-8822-b8cb8bea5266 admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:17.447816 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33078e03-5049-4daa-8822-b8cb8bea5266 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:17.460603 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33078e03-5049-4daa-8822-b8cb8bea5266 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:17.462179 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:35:17.578193 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-a7185332-8a5b-4c24-b1f9-022ce3819479 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '050826b30ece4dda86e5049bfd6c6d01'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:17.594332 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-a7185332-8a5b-4c24-b1f9-022ce3819479 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '050826b30ece4dda86e5049bfd6c6d01'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:17.595610 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-a7185332-8a5b-4c24-b1f9-022ce3819479 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '050826b30ece4dda86e5049bfd6c6d01'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:17.606221 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 829/1679] 10.222.0.66 () {60 vars in 1152 bytes} [Mon Jul 18 02:35:17 2022] POST /identity/v3/projects => generated 373 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 02:35:17.612251 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33078e03-5049-4daa-8822-b8cb8bea5266 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:17.616911 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-33078e03-5049-4daa-8822-b8cb8bea5266 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:17.641293 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-a7185332-8a5b-4c24-b1f9-022ce3819479 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '050826b30ece4dda86e5049bfd6c6d01'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:17.658198 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 851/1680] 10.222.0.66 () {58 vars in 1223 bytes} [Mon Jul 18 02:35:17 2022] DELETE /identity/v3/users/050826b30ece4dda86e5049bfd6c6d01 => generated 0 bytes in 551 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:35:17.745513 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33078e03-5049-4daa-8822-b8cb8bea5266 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:17.915774 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0aa6af1-8042-47ae-87da-23454d5f73a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:17.916214 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aa6af1-8042-47ae-87da-23454d5f73a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:17.916547 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aa6af1-8042-47ae-87da-23454d5f73a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:17.916852 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aa6af1-8042-47ae-87da-23454d5f73a7 admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:17.917894 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0aa6af1-8042-47ae-87da-23454d5f73a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:17.919533 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0aa6af1-8042-47ae-87da-23454d5f73a7 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:18.104780 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 830/1681] 10.222.0.66 () {60 vars in 1143 bytes} [Mon Jul 18 02:35:17 2022] POST /identity/v3/users => generated 287 bytes in 494 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 02:35:18.110535 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7185332-8a5b-4c24-b1f9-022ce3819479 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:18.128646 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7185332-8a5b-4c24-b1f9-022ce3819479 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:18.139434 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c66d98-f904-4d3c-912e-4e22e9dca560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:18.141609 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c66d98-f904-4d3c-912e-4e22e9dca560 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:18.141609 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c66d98-f904-4d3c-912e-4e22e9dca560 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:18.141609 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c66d98-f904-4d3c-912e-4e22e9dca560 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:18.142551 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c66d98-f904-4d3c-912e-4e22e9dca560 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:18.144692 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c66d98-f904-4d3c-912e-4e22e9dca560 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:18.155645 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0aa6af1-8042-47ae-87da-23454d5f73a7 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:18.157446 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-d0aa6af1-8042-47ae-87da-23454d5f73a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:18.175030 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 852/1682] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:35:18 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:35:18.180494 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0aa6af1-8042-47ae-87da-23454d5f73a7 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:18.181739 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c66d98-f904-4d3c-912e-4e22e9dca560 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:18.191199 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138bc3cf-a44b-489d-a2cd-2c43c577177f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:18.192080 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138bc3cf-a44b-489d-a2cd-2c43c577177f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:18.192364 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138bc3cf-a44b-489d-a2cd-2c43c577177f admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:18.192683 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138bc3cf-a44b-489d-a2cd-2c43c577177f admin admin] PATH_INFO: `/v3/projects/117f57edd1524d03b049a87c6b8bd946` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:18.194126 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138bc3cf-a44b-489d-a2cd-2c43c577177f admin admin] RBAC: Authorizing `identity:delete_project(project_id=117f57edd1524d03b049a87c6b8bd946)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:18.199605 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138bc3cf-a44b-489d-a2cd-2c43c577177f admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:18.206881 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c66d98-f904-4d3c-912e-4e22e9dca560 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:18.235418 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306819f3-8629-42da-9e03-813f6724294f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:18.236344 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306819f3-8629-42da-9e03-813f6724294f admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:18.236641 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306819f3-8629-42da-9e03-813f6724294f admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:18.236926 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306819f3-8629-42da-9e03-813f6724294f admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:18.241860 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306819f3-8629-42da-9e03-813f6724294f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:18.243785 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306819f3-8629-42da-9e03-813f6724294f admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:18.294588 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 853/1683] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:35:18 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:35:18.302589 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306819f3-8629-42da-9e03-813f6724294f admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:18.312015 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:35:18.332350 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306819f3-8629-42da-9e03-813f6724294f admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:18.356366 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791bb764-acb1-4dc3-9ab3-1e86d5c384ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:18.357393 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791bb764-acb1-4dc3-9ab3-1e86d5c384ba admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:18.357994 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791bb764-acb1-4dc3-9ab3-1e86d5c384ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:18.358643 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791bb764-acb1-4dc3-9ab3-1e86d5c384ba admin admin] PATH_INFO: `/v3/projects/111367517d7f4746b244db54a7af290a/users/92b24ef8f7f44c87a55d1410d3cb07d4/roles/65feabf3b27944a8b01a86c8ac608457` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:18.387576 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791bb764-acb1-4dc3-9ab3-1e86d5c384ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=111367517d7f4746b244db54a7af290a, user_id=92b24ef8f7f44c87a55d1410d3cb07d4, role_id=65feabf3b27944a8b01a86c8ac608457)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:18.391765 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791bb764-acb1-4dc3-9ab3-1e86d5c384ba admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:18.655544 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 854/1684] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:35:18 2022] PUT /identity/v3/projects/111367517d7f4746b244db54a7af290a/users/92b24ef8f7f44c87a55d1410d3cb07d4/roles/65feabf3b27944a8b01a86c8ac608457 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:35:18.655544 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791bb764-acb1-4dc3-9ab3-1e86d5c384ba admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:18.662039 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-791bb764-acb1-4dc3-9ab3-1e86d5c384ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:18.670389 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-138bc3cf-a44b-489d-a2cd-2c43c577177f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '117f57edd1524d03b049a87c6b8bd946', 'request_id': 'req-138bc3cf-a44b-489d-a2cd-2c43c577177f'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:18.678294 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-138bc3cf-a44b-489d-a2cd-2c43c577177f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '117f57edd1524d03b049a87c6b8bd946', 'request_id': 'req-138bc3cf-a44b-489d-a2cd-2c43c577177f'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:18.685337 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-138bc3cf-a44b-489d-a2cd-2c43c577177f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '117f57edd1524d03b049a87c6b8bd946', 'request_id': 'req-138bc3cf-a44b-489d-a2cd-2c43c577177f'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:18.762362 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-138bc3cf-a44b-489d-a2cd-2c43c577177f admin admin] The token cache is being invalidate because project 117f57edd1524d03b049a87c6b8bd946 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 02:35:18.764491 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-138bc3cf-a44b-489d-a2cd-2c43c577177f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:18.764491 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-138bc3cf-a44b-489d-a2cd-2c43c577177f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:18.769580 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 831/1685] 10.222.0.66 () {58 vars in 1232 bytes} [Mon Jul 18 02:35:18 2022] DELETE /identity/v3/projects/117f57edd1524d03b049a87c6b8bd946 => generated 0 bytes in 612 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 02:35:18.771552 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2e1747-a737-4ecc-bc15-d3fc6ac1deb2 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:18.773859 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2e1747-a737-4ecc-bc15-d3fc6ac1deb2 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:18.774924 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2e1747-a737-4ecc-bc15-d3fc6ac1deb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:18.832493 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791bb764-acb1-4dc3-9ab3-1e86d5c384ba admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:18.839029 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-0c2e1747-a737-4ecc-bc15-d3fc6ac1deb2 None None] MFA Rules not processed for user `92b24ef8f7f44c87a55d1410d3cb07d4`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:35:18.843497 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-0c2e1747-a737-4ecc-bc15-d3fc6ac1deb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:19.027020 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 832/1686] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:35:18 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:35:19.030634 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06648cee-65d8-48f8-888e-adc19573564f None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:19.031527 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06648cee-65d8-48f8-888e-adc19573564f None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:19.031527 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06648cee-65d8-48f8-888e-adc19573564f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:19.057066 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bee2f64-fa20-4e21-b22a-3cd5d52f8cc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:19.067047 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bee2f64-fa20-4e21-b22a-3cd5d52f8cc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:19.075011 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-06648cee-65d8-48f8-888e-adc19573564f None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:35:19.078519 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bee2f64-fa20-4e21-b22a-3cd5d52f8cc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:19.088239 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bee2f64-fa20-4e21-b22a-3cd5d52f8cc1 admin admin] PATH_INFO: `/v3/projects/d833251e6cee4ce396d02f001221c99b` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:19.088543 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-06648cee-65d8-48f8-888e-adc19573564f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:19.088653 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bee2f64-fa20-4e21-b22a-3cd5d52f8cc1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d833251e6cee4ce396d02f001221c99b)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:19.088653 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bee2f64-fa20-4e21-b22a-3cd5d52f8cc1 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:19.102033 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 833/1687] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:35:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:35:19.102859 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06648cee-65d8-48f8-888e-adc19573564f None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:19.108009 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:35:19.110289 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-06648cee-65d8-48f8-888e-adc19573564f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:19.362933 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06648cee-65d8-48f8-888e-adc19573564f None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:19.372207 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-8bee2f64-fa20-4e21-b22a-3cd5d52f8cc1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd833251e6cee4ce396d02f001221c99b', 'request_id': 'req-8bee2f64-fa20-4e21-b22a-3cd5d52f8cc1'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:19.384217 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-8bee2f64-fa20-4e21-b22a-3cd5d52f8cc1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd833251e6cee4ce396d02f001221c99b', 'request_id': 'req-8bee2f64-fa20-4e21-b22a-3cd5d52f8cc1'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:19.472303 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-8bee2f64-fa20-4e21-b22a-3cd5d52f8cc1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd833251e6cee4ce396d02f001221c99b', 'request_id': 'req-8bee2f64-fa20-4e21-b22a-3cd5d52f8cc1'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:19.498804 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bc6fdd-3e9a-4762-8632-0f01c6f19b8c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80b138831bc54c7b986d50092d62d4af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:19.499485 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bc6fdd-3e9a-4762-8632-0f01c6f19b8c service nova] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:19.499919 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bc6fdd-3e9a-4762-8632-0f01c6f19b8c service nova] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:19.500200 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bc6fdd-3e9a-4762-8632-0f01c6f19b8c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:19.515089 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-8bee2f64-fa20-4e21-b22a-3cd5d52f8cc1 admin admin] The token cache is being invalidate because project d833251e6cee4ce396d02f001221c99b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 02:35:19.515724 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-8bee2f64-fa20-4e21-b22a-3cd5d52f8cc1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:19.516711 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-8bee2f64-fa20-4e21-b22a-3cd5d52f8cc1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:19.519502 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 855/1688] 10.222.0.66 () {58 vars in 1232 bytes} [Mon Jul 18 02:35:18 2022] DELETE /identity/v3/projects/d833251e6cee4ce396d02f001221c99b => generated 0 bytes in 868 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:35:19.521137 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662e237a-5af6-4592-b043-fdccc7c6f5da None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:19.521906 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662e237a-5af6-4592-b043-fdccc7c6f5da None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:19.522480 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662e237a-5af6-4592-b043-fdccc7c6f5da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:19.558318 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-662e237a-5af6-4592-b043-fdccc7c6f5da None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:35:19.564396 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-662e237a-5af6-4592-b043-fdccc7c6f5da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:19.666829 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bc6fdd-3e9a-4762-8632-0f01c6f19b8c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:19.666829 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bc6fdd-3e9a-4762-8632-0f01c6f19b8c service nova] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:19.669164 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 834/1689] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:35:19 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 567 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:35:19.703564 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 856/1690] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:35:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:35:19.704352 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910fd370-fe1c-4766-ac86-541e67bb0bcf None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:19.705350 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910fd370-fe1c-4766-ac86-541e67bb0bcf None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:19.706910 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910fd370-fe1c-4766-ac86-541e67bb0bcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:19.707333 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662e237a-5af6-4592-b043-fdccc7c6f5da None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:19.717670 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662e237a-5af6-4592-b043-fdccc7c6f5da None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:19.733668 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc0b812-86ca-4066-aafe-5f3b518bdc17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:19.734905 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc0b812-86ca-4066-aafe-5f3b518bdc17 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:19.735204 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc0b812-86ca-4066-aafe-5f3b518bdc17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:19.735536 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc0b812-86ca-4066-aafe-5f3b518bdc17 admin admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:19.736520 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc0b812-86ca-4066-aafe-5f3b518bdc17 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:35:19.736820 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc0b812-86ca-4066-aafe-5f3b518bdc17 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:19.738473 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc0b812-86ca-4066-aafe-5f3b518bdc17 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:19.759698 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-910fd370-fe1c-4766-ac86-541e67bb0bcf None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:35:19.759698 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-910fd370-fe1c-4766-ac86-541e67bb0bcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:19.790129 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 857/1691] 10.222.0.66 () {58 vars in 1152 bytes} [Mon Jul 18 02:35:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:35:19.792155 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc0b812-86ca-4066-aafe-5f3b518bdc17 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:19.807840 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 835/1692] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:35:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:35:19.809330 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02993d71-ffef-49fd-bf72-b974e0cb29b3 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:19.809629 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02993d71-ffef-49fd-bf72-b974e0cb29b3 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:19.809968 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02993d71-ffef-49fd-bf72-b974e0cb29b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:19.827318 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc0b812-86ca-4066-aafe-5f3b518bdc17 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:19.842509 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a3dfb0-6191-4aef-955a-c35a5beaf157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:19.854228 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a3dfb0-6191-4aef-955a-c35a5beaf157 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:19.854567 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a3dfb0-6191-4aef-955a-c35a5beaf157 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:19.854858 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a3dfb0-6191-4aef-955a-c35a5beaf157 admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:19.867089 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-02993d71-ffef-49fd-bf72-b974e0cb29b3 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:35:19.868325 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-02993d71-ffef-49fd-bf72-b974e0cb29b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:19.879118 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a3dfb0-6191-4aef-955a-c35a5beaf157 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:19.885857 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a3dfb0-6191-4aef-955a-c35a5beaf157 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:19.891015 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 836/1693] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:35:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:35:19.900624 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02993d71-ffef-49fd-bf72-b974e0cb29b3 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:19.914944 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02993d71-ffef-49fd-bf72-b974e0cb29b3 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:19.920857 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0a405d-3021-4d20-9c67-db59c3159d8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:19.921266 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0a405d-3021-4d20-9c67-db59c3159d8d admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:19.921547 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0a405d-3021-4d20-9c67-db59c3159d8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:19.921825 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0a405d-3021-4d20-9c67-db59c3159d8d admin admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:19.922638 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0a405d-3021-4d20-9c67-db59c3159d8d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:35:19.922945 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0a405d-3021-4d20-9c67-db59c3159d8d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:19.924643 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0a405d-3021-4d20-9c67-db59c3159d8d admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:19.937370 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:35:19.954168 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 837/1694] 10.222.0.66 () {58 vars in 1152 bytes} [Mon Jul 18 02:35:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:35:19.956339 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea260a07-efa4-4f24-b687-a8bf86e4f906 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:19.956339 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea260a07-efa4-4f24-b687-a8bf86e4f906 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:19.956339 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea260a07-efa4-4f24-b687-a8bf86e4f906 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:19.974033 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-ea260a07-efa4-4f24-b687-a8bf86e4f906 None None] MFA Rules not processed for user `879f7a4cf6664e76bddffc804658b250`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:35:19.976886 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-ea260a07-efa4-4f24-b687-a8bf86e4f906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:20.064929 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 858/1695] 10.222.0.66 () {60 vars in 1152 bytes} [Mon Jul 18 02:35:19 2022] POST /identity/v3/projects => generated 371 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:35:20.069426 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a3dfb0-6191-4aef-955a-c35a5beaf157 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:20.083911 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 838/1696] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:35:19 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:35:20.091872 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea260a07-efa4-4f24-b687-a8bf86e4f906 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:20.108020 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea260a07-efa4-4f24-b687-a8bf86e4f906 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:20.218268 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a3dfb0-6191-4aef-955a-c35a5beaf157 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:20.222469 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afebde3a-4a15-4af1-9db1-54980a59d742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:20.229048 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afebde3a-4a15-4af1-9db1-54980a59d742 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:20.229671 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afebde3a-4a15-4af1-9db1-54980a59d742 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:20.230178 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afebde3a-4a15-4af1-9db1-54980a59d742 admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:20.231125 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afebde3a-4a15-4af1-9db1-54980a59d742 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:20.234101 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afebde3a-4a15-4af1-9db1-54980a59d742 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:20.287609 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f980315-5b01-42ea-a088-e1d51c9e97b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:20.288719 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f980315-5b01-42ea-a088-e1d51c9e97b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:20.289174 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f980315-5b01-42ea-a088-e1d51c9e97b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:20.289606 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f980315-5b01-42ea-a088-e1d51c9e97b1 admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:20.290493 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f980315-5b01-42ea-a088-e1d51c9e97b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:20.298419 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f980315-5b01-42ea-a088-e1d51c9e97b1 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:20.406602 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 839/1697] 10.222.0.66 () {60 vars in 1143 bytes} [Mon Jul 18 02:35:20 2022] POST /identity/v3/users => generated 286 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:35:20.410223 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 859/1698] 10.222.0.66 () {60 vars in 1152 bytes} [Mon Jul 18 02:35:20 2022] POST /identity/v3/projects => generated 373 bytes in 346 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:35:20.410822 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f980315-5b01-42ea-a088-e1d51c9e97b1 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:20.414141 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afebde3a-4a15-4af1-9db1-54980a59d742 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:20.430964 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f980315-5b01-42ea-a088-e1d51c9e97b1 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:20.523106 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afebde3a-4a15-4af1-9db1-54980a59d742 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:20.529048 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b1a120-2c59-4540-b4b7-c201c7c4ff0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:20.530249 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b1a120-2c59-4540-b4b7-c201c7c4ff0b admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:20.530682 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b1a120-2c59-4540-b4b7-c201c7c4ff0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:20.531061 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b1a120-2c59-4540-b4b7-c201c7c4ff0b admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:20.531876 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b1a120-2c59-4540-b4b7-c201c7c4ff0b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:20.534115 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b1a120-2c59-4540-b4b7-c201c7c4ff0b admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:20.538777 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cdbd95-be39-4e17-b6ff-0c0162ce95a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:20.539813 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cdbd95-be39-4e17-b6ff-0c0162ce95a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:20.539953 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cdbd95-be39-4e17-b6ff-0c0162ce95a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:20.540160 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cdbd95-be39-4e17-b6ff-0c0162ce95a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:20.541268 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cdbd95-be39-4e17-b6ff-0c0162ce95a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:20.543077 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cdbd95-be39-4e17-b6ff-0c0162ce95a2 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:20.559277 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 840/1699] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:35:20 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:35:20.561413 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cdbd95-be39-4e17-b6ff-0c0162ce95a2 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:20.572789 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cdbd95-be39-4e17-b6ff-0c0162ce95a2 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:20.579361 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4f6b49-cc94-4269-919e-c52c927a0252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:20.580061 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4f6b49-cc94-4269-919e-c52c927a0252 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:20.580353 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4f6b49-cc94-4269-919e-c52c927a0252 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:20.580682 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4f6b49-cc94-4269-919e-c52c927a0252 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:20.581600 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4f6b49-cc94-4269-919e-c52c927a0252 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:20.594315 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4f6b49-cc94-4269-919e-c52c927a0252 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:20.611027 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 841/1700] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:35:20 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:35:20.625609 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4f6b49-cc94-4269-919e-c52c927a0252 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:20.626675 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4f6b49-cc94-4269-919e-c52c927a0252 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:20.631413 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d5887f-1a0e-4df9-9b1b-b32c835cb747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:20.632479 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d5887f-1a0e-4df9-9b1b-b32c835cb747 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:20.632864 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d5887f-1a0e-4df9-9b1b-b32c835cb747 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:20.633146 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d5887f-1a0e-4df9-9b1b-b32c835cb747 admin admin] PATH_INFO: `/v3/projects/77733b7b070540db93d486df83e565f1/users/abcb0a70133544c693cf846812198893/roles/65feabf3b27944a8b01a86c8ac608457` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:20.657165 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d5887f-1a0e-4df9-9b1b-b32c835cb747 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77733b7b070540db93d486df83e565f1, user_id=abcb0a70133544c693cf846812198893, role_id=65feabf3b27944a8b01a86c8ac608457)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:20.657165 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d5887f-1a0e-4df9-9b1b-b32c835cb747 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:20.661769 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 860/1701] 10.222.0.66 () {60 vars in 1143 bytes} [Mon Jul 18 02:35:20 2022] POST /identity/v3/users => generated 287 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 02:35:20.665540 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b1a120-2c59-4540-b4b7-c201c7c4ff0b admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:20.686717 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b1a120-2c59-4540-b4b7-c201c7c4ff0b admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:20.699026 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3e3a6f-551d-4fc3-bf7b-8836c8f4f0fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:20.699730 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3e3a6f-551d-4fc3-bf7b-8836c8f4f0fe admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:20.700019 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3e3a6f-551d-4fc3-bf7b-8836c8f4f0fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:20.700300 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3e3a6f-551d-4fc3-bf7b-8836c8f4f0fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:20.701587 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3e3a6f-551d-4fc3-bf7b-8836c8f4f0fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:20.714379 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3e3a6f-551d-4fc3-bf7b-8836c8f4f0fe admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:20.736060 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 861/1702] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:35:20 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 02:35:20.740010 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3e3a6f-551d-4fc3-bf7b-8836c8f4f0fe admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:20.743753 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 842/1703] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:35:20 2022] PUT /identity/v3/projects/77733b7b070540db93d486df83e565f1/users/abcb0a70133544c693cf846812198893/roles/65feabf3b27944a8b01a86c8ac608457 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:35:20.753308 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa6fd02-cc0c-432c-81f3-653dc3c4f19b None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:20.753669 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa6fd02-cc0c-432c-81f3-653dc3c4f19b None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:20.754013 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa6fd02-cc0c-432c-81f3-653dc3c4f19b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:20.759405 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3e3a6f-551d-4fc3-bf7b-8836c8f4f0fe admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:20.801628 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-faa6fd02-cc0c-432c-81f3-653dc3c4f19b None None] MFA Rules not processed for user `abcb0a70133544c693cf846812198893`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:35:20.802908 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-faa6fd02-cc0c-432c-81f3-653dc3c4f19b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:20.952160 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06914edc-1c83-4655-8504-10682f8deeb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:20.952860 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06914edc-1c83-4655-8504-10682f8deeb7 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:20.953151 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06914edc-1c83-4655-8504-10682f8deeb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:20.953426 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06914edc-1c83-4655-8504-10682f8deeb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:20.957325 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06914edc-1c83-4655-8504-10682f8deeb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:20.967850 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06914edc-1c83-4655-8504-10682f8deeb7 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:20.998422 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 862/1704] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:35:20 2022] GET /identity/v3/roles => generated 1420 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:35:20.998422 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06914edc-1c83-4655-8504-10682f8deeb7 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:21.011584 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 843/1705] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:35:20 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 02:35:21.011783 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06914edc-1c83-4655-8504-10682f8deeb7 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:21.017842 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce63680-891c-4fb0-8cb3-87428a01268b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:21.018947 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce63680-891c-4fb0-8cb3-87428a01268b admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:21.019335 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce63680-891c-4fb0-8cb3-87428a01268b admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:21.020625 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce63680-891c-4fb0-8cb3-87428a01268b admin admin] PATH_INFO: `/v3/projects/9917e65b8ee449038cbd72443c368db7/users/77989d2930274bd2832378750f807683/roles/65feabf3b27944a8b01a86c8ac608457` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:21.039567 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce63680-891c-4fb0-8cb3-87428a01268b admin admin] RBAC: Authorizing `identity:create_grant(project_id=9917e65b8ee449038cbd72443c368db7, user_id=77989d2930274bd2832378750f807683, role_id=65feabf3b27944a8b01a86c8ac608457)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:21.047382 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce63680-891c-4fb0-8cb3-87428a01268b admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:21.070460 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:35:21.152725 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 863/1706] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:35:20 2022] PUT /identity/v3/projects/9917e65b8ee449038cbd72443c368db7/users/77989d2930274bd2832378750f807683/roles/65feabf3b27944a8b01a86c8ac608457 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 02:35:21.166421 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54616b92-fbba-445c-8b06-0c69677de9e9 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:21.166723 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54616b92-fbba-445c-8b06-0c69677de9e9 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:21.166989 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54616b92-fbba-445c-8b06-0c69677de9e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:21.232625 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce63680-891c-4fb0-8cb3-87428a01268b admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:21.235489 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-2ce63680-891c-4fb0-8cb3-87428a01268b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:21.236046 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-54616b92-fbba-445c-8b06-0c69677de9e9 None None] MFA Rules not processed for user `77989d2930274bd2832378750f807683`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:35:21.237746 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-54616b92-fbba-445c-8b06-0c69677de9e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:21.370068 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 844/1707] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:35:21 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:35:21.370068 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9489da3d-25b3-4d17-93f8-72b2c0da631e None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:21.370068 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9489da3d-25b3-4d17-93f8-72b2c0da631e None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:21.370068 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9489da3d-25b3-4d17-93f8-72b2c0da631e None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:21.373827 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 845/1708] 10.222.0.66 () {54 vars in 912 bytes} [Mon Jul 18 02:35:21 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:35:21.375576 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9489da3d-25b3-4d17-93f8-72b2c0da631e None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:21.404795 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce63680-891c-4fb0-8cb3-87428a01268b admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:21.426487 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9988cfeb-cc96-4566-92e3-d7e78fe8a3cf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80b138831bc54c7b986d50092d62d4af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:21.426487 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9988cfeb-cc96-4566-92e3-d7e78fe8a3cf service nova] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:21.427151 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9988cfeb-cc96-4566-92e3-d7e78fe8a3cf service nova] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:21.442266 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9988cfeb-cc96-4566-92e3-d7e78fe8a3cf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:21.528302 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9489da3d-25b3-4d17-93f8-72b2c0da631e None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:21.540664 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffc824d-1637-4c28-8496-a98bb2285e7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:21.541328 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffc824d-1637-4c28-8496-a98bb2285e7b admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:21.541620 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffc824d-1637-4c28-8496-a98bb2285e7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:21.541921 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffc824d-1637-4c28-8496-a98bb2285e7b admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:21.542906 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffc824d-1637-4c28-8496-a98bb2285e7b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:21.545071 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffc824d-1637-4c28-8496-a98bb2285e7b admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:21.572432 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9988cfeb-cc96-4566-92e3-d7e78fe8a3cf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:21.574802 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9988cfeb-cc96-4566-92e3-d7e78fe8a3cf service nova] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:21.596027 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 864/1709] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:35:21 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 02:35:21.597225 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44010e23-1e4a-4fa3-ab78-64af7fd60bc2 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:21.598110 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44010e23-1e4a-4fa3-ab78-64af7fd60bc2 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:21.600851 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44010e23-1e4a-4fa3-ab78-64af7fd60bc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:21.646727 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 846/1710] 10.222.0.66 () {60 vars in 1152 bytes} [Mon Jul 18 02:35:21 2022] POST /identity/v3/projects => generated 373 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:35:21.651900 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffc824d-1637-4c28-8496-a98bb2285e7b admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:21.654201 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-44010e23-1e4a-4fa3-ab78-64af7fd60bc2 None None] MFA Rules not processed for user `eaf19a259c2741cea375a38763e33cdc`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:35:21.657205 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-44010e23-1e4a-4fa3-ab78-64af7fd60bc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:21.796862 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 865/1711] 10.222.0.66 () {58 vars in 1009 bytes} [Mon Jul 18 02:35:21 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:35:21.800397 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffc824d-1637-4c28-8496-a98bb2285e7b admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:21.813925 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66702bf3-bc80-4410-aaf9-cee9fb58d049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:21.814954 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66702bf3-bc80-4410-aaf9-cee9fb58d049 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:21.814954 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66702bf3-bc80-4410-aaf9-cee9fb58d049 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:21.814954 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66702bf3-bc80-4410-aaf9-cee9fb58d049 admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:21.817075 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66702bf3-bc80-4410-aaf9-cee9fb58d049 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:21.822987 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44010e23-1e4a-4fa3-ab78-64af7fd60bc2 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:21.825245 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-44010e23-1e4a-4fa3-ab78-64af7fd60bc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:21.833875 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66702bf3-bc80-4410-aaf9-cee9fb58d049 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:21.843746 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44010e23-1e4a-4fa3-ab78-64af7fd60bc2 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:21.863081 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c02b29-494b-45cc-bd95-733aa050adfd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eaf19a259c2741cea375a38763e33cdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:21.863081 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c02b29-494b-45cc-bd95-733aa050adfd service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:21.863081 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c02b29-494b-45cc-bd95-733aa050adfd service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:21.863081 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c02b29-494b-45cc-bd95-733aa050adfd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:21.884050 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c02b29-494b-45cc-bd95-733aa050adfd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:21.887337 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c02b29-494b-45cc-bd95-733aa050adfd service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:21.911676 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 866/1712] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:35:21 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 02:35:21.959909 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c02b29-494b-45cc-bd95-733aa050adfd service neutron] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:21.991658 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c02b29-494b-45cc-bd95-733aa050adfd service neutron] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:22.002276 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf556e8e-5a7d-43ce-adb7-883ce3a61dc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:22.003942 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 847/1713] 10.222.0.66 () {60 vars in 1143 bytes} [Mon Jul 18 02:35:21 2022] POST /identity/v3/users => generated 293 bytes in 351 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:35:22.006006 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf556e8e-5a7d-43ce-adb7-883ce3a61dc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:22.006481 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf556e8e-5a7d-43ce-adb7-883ce3a61dc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:22.006944 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf556e8e-5a7d-43ce-adb7-883ce3a61dc7 admin admin] PATH_INFO: `/v3/users/abcb0a70133544c693cf846812198893` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:22.008850 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf556e8e-5a7d-43ce-adb7-883ce3a61dc7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=abcb0a70133544c693cf846812198893)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:22.010980 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66702bf3-bc80-4410-aaf9-cee9fb58d049 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:22.015125 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf556e8e-5a7d-43ce-adb7-883ce3a61dc7 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:22.067113 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66702bf3-bc80-4410-aaf9-cee9fb58d049 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:22.067113 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a59f796-162f-4403-9aa0-14a57bace17b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:22.067113 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a59f796-162f-4403-9aa0-14a57bace17b admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:22.067113 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a59f796-162f-4403-9aa0-14a57bace17b admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:22.067645 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a59f796-162f-4403-9aa0-14a57bace17b admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:22.067645 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a59f796-162f-4403-9aa0-14a57bace17b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:22.067645 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a59f796-162f-4403-9aa0-14a57bace17b admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:22.076906 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 848/1714] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:35:22 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:35:22.084108 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a59f796-162f-4403-9aa0-14a57bace17b admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:22.098887 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a59f796-162f-4403-9aa0-14a57bace17b admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:22.109409 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d28a9c-e9e6-4ad9-9676-b2bd964570cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:22.109409 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d28a9c-e9e6-4ad9-9676-b2bd964570cc admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:22.109409 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d28a9c-e9e6-4ad9-9676-b2bd964570cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:22.109409 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d28a9c-e9e6-4ad9-9676-b2bd964570cc admin admin] PATH_INFO: `/v3/projects/a1adfe26f31649cab9688e5c766ed9da/users/87fcd3f1c4fa41fe94a59def1e79541e/roles/d4aa3324f0c84887b41d0dc4acf7f30e` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:22.152530 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d28a9c-e9e6-4ad9-9676-b2bd964570cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1adfe26f31649cab9688e5c766ed9da, user_id=87fcd3f1c4fa41fe94a59def1e79541e, role_id=d4aa3324f0c84887b41d0dc4acf7f30e)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:22.159019 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d28a9c-e9e6-4ad9-9676-b2bd964570cc admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:22.310739 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 849/1715] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:35:22 2022] PUT /identity/v3/projects/a1adfe26f31649cab9688e5c766ed9da/users/87fcd3f1c4fa41fe94a59def1e79541e/roles/d4aa3324f0c84887b41d0dc4acf7f30e => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:35:22.312735 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d28a9c-e9e6-4ad9-9676-b2bd964570cc admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:22.433560 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-bf556e8e-5a7d-43ce-adb7-883ce3a61dc7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'abcb0a70133544c693cf846812198893'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:22.454144 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d28a9c-e9e6-4ad9-9676-b2bd964570cc admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:22.472072 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874f60f8-e787-4b42-98b4-3e555ed0de68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:22.472072 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874f60f8-e787-4b42-98b4-3e555ed0de68 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:22.472072 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874f60f8-e787-4b42-98b4-3e555ed0de68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:22.472072 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874f60f8-e787-4b42-98b4-3e555ed0de68 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:22.472459 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874f60f8-e787-4b42-98b4-3e555ed0de68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:22.473510 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874f60f8-e787-4b42-98b4-3e555ed0de68 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:22.489850 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 850/1716] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:35:22 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:35:22.490924 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874f60f8-e787-4b42-98b4-3e555ed0de68 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:22.493041 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-bf556e8e-5a7d-43ce-adb7-883ce3a61dc7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'abcb0a70133544c693cf846812198893'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:22.503926 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874f60f8-e787-4b42-98b4-3e555ed0de68 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:22.510222 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2ce8da-187a-4e06-8517-61104af42b73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:22.511031 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2ce8da-187a-4e06-8517-61104af42b73 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:22.511315 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2ce8da-187a-4e06-8517-61104af42b73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:22.511592 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2ce8da-187a-4e06-8517-61104af42b73 admin admin] PATH_INFO: `/v3/projects/a1adfe26f31649cab9688e5c766ed9da/users/87fcd3f1c4fa41fe94a59def1e79541e/roles/d4aa3324f0c84887b41d0dc4acf7f30e` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:22.523342 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2ce8da-187a-4e06-8517-61104af42b73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1adfe26f31649cab9688e5c766ed9da, user_id=87fcd3f1c4fa41fe94a59def1e79541e, role_id=d4aa3324f0c84887b41d0dc4acf7f30e)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:22.527304 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2ce8da-187a-4e06-8517-61104af42b73 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:22.529693 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-bf556e8e-5a7d-43ce-adb7-883ce3a61dc7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'abcb0a70133544c693cf846812198893'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:22.530209 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-bf556e8e-5a7d-43ce-adb7-883ce3a61dc7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'abcb0a70133544c693cf846812198893'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:22.564973 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 851/1717] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:35:22 2022] PUT /identity/v3/projects/a1adfe26f31649cab9688e5c766ed9da/users/87fcd3f1c4fa41fe94a59def1e79541e/roles/d4aa3324f0c84887b41d0dc4acf7f30e => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:35:22.566591 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13d6e6a-1f53-4f2d-9771-a5dd27303dde None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:22.566884 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13d6e6a-1f53-4f2d-9771-a5dd27303dde None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:22.571751 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13d6e6a-1f53-4f2d-9771-a5dd27303dde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:22.583718 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-bf556e8e-5a7d-43ce-adb7-883ce3a61dc7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'abcb0a70133544c693cf846812198893'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:22.593478 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 867/1718] 10.222.0.66 () {58 vars in 1223 bytes} [Mon Jul 18 02:35:21 2022] DELETE /identity/v3/users/abcb0a70133544c693cf846812198893 => generated 0 bytes in 635 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:35:22.596686 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:35:22.606237 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c75d80-d473-416d-8b28-171db9bd94c2 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:22.606594 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c75d80-d473-416d-8b28-171db9bd94c2 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:22.608628 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c75d80-d473-416d-8b28-171db9bd94c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:22.635307 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-b1c75d80-d473-416d-8b28-171db9bd94c2 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:35:22.636534 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-b1c75d80-d473-416d-8b28-171db9bd94c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:22.647319 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-c13d6e6a-1f53-4f2d-9771-a5dd27303dde None None] MFA Rules not processed for user `87fcd3f1c4fa41fe94a59def1e79541e`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:35:22.648739 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-c13d6e6a-1f53-4f2d-9771-a5dd27303dde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:22.766378 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 868/1719] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:35:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:35:22.767502 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc19b76-b085-4f3b-94a9-b7922214ea2c None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:22.767846 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc19b76-b085-4f3b-94a9-b7922214ea2c None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:22.768129 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc19b76-b085-4f3b-94a9-b7922214ea2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:22.821199 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 852/1720] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:35:22 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:35:22.824310 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13d6e6a-1f53-4f2d-9771-a5dd27303dde None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:22.827778 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-c13d6e6a-1f53-4f2d-9771-a5dd27303dde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:22.881287 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-1bc19b76-b085-4f3b-94a9-b7922214ea2c None None] MFA Rules not processed for user `768f872096584d4093fc616d547aabb6`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:35:22.882820 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-1bc19b76-b085-4f3b-94a9-b7922214ea2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:22.934649 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13d6e6a-1f53-4f2d-9771-a5dd27303dde None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:22.942228 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d96eb08-b331-40ee-871c-b0defe1c2ca3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eaf19a259c2741cea375a38763e33cdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:22.947358 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d96eb08-b331-40ee-871c-b0defe1c2ca3 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:22.947739 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d96eb08-b331-40ee-871c-b0defe1c2ca3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:22.948013 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d96eb08-b331-40ee-871c-b0defe1c2ca3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:22.962248 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d96eb08-b331-40ee-871c-b0defe1c2ca3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:22.964351 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d96eb08-b331-40ee-871c-b0defe1c2ca3 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:22.971952 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 869/1721] 10.222.0.66 () {58 vars in 1009 bytes} [Mon Jul 18 02:35:22 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:35:22.974204 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc19b76-b085-4f3b-94a9-b7922214ea2c None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:22.974688 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-1bc19b76-b085-4f3b-94a9-b7922214ea2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:22.991681 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 853/1722] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:35:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:35:23.147288 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc19b76-b085-4f3b-94a9-b7922214ea2c None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:23.153582 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff03a12-610a-46fc-89ca-b64a48609245 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd49388facdf446885e55b45ef91f01e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:23.155693 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff03a12-610a-46fc-89ca-b64a48609245 service cinder] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:23.156182 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff03a12-610a-46fc-89ca-b64a48609245 service cinder] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:23.156664 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff03a12-610a-46fc-89ca-b64a48609245 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:23.189195 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff03a12-610a-46fc-89ca-b64a48609245 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:23.192850 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff03a12-610a-46fc-89ca-b64a48609245 service cinder] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:23.286337 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 870/1723] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:35:22 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:35:23.497383 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d96eb08-b331-40ee-871c-b0defe1c2ca3 service neutron] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:23.499998 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-4d96eb08-b331-40ee-871c-b0defe1c2ca3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:23.621327 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a8f27a-012f-4906-b77b-77d0fb528bc6 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:23.621327 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a8f27a-012f-4906-b77b-77d0fb528bc6 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:23.621972 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a8f27a-012f-4906-b77b-77d0fb528bc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:23.658391 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-69a8f27a-012f-4906-b77b-77d0fb528bc6 None None] MFA Rules not processed for user `94c16391d39f4498a55c89bf2838788f`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:35:23.663529 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-69a8f27a-012f-4906-b77b-77d0fb528bc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:23.732344 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:35:23.801818 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d96eb08-b331-40ee-871c-b0defe1c2ca3 service neutron] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:23.811420 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7ecf83-bb84-497e-b986-8034c6a63186 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4300622cbd347dd8f5227bb960e98d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:23.812876 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7ecf83-bb84-497e-b986-8034c6a63186 service swift] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:23.813378 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7ecf83-bb84-497e-b986-8034c6a63186 service swift] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:23.813891 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7ecf83-bb84-497e-b986-8034c6a63186 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:23.838389 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 871/1724] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:35:23 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:35:23.887659 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7ecf83-bb84-497e-b986-8034c6a63186 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:23.892622 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7ecf83-bb84-497e-b986-8034c6a63186 service swift] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:23.929372 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770f57ea-00aa-4c7f-901d-3c7fa5ed42be None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:23.929702 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770f57ea-00aa-4c7f-901d-3c7fa5ed42be None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:23.929990 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770f57ea-00aa-4c7f-901d-3c7fa5ed42be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:23.957162 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 854/1725] 10.222.0.66 () {60 vars in 1384 bytes} [Mon Jul 18 02:35:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 18 02:35:23.988144 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-770f57ea-00aa-4c7f-901d-3c7fa5ed42be None None] MFA Rules not processed for user `7ddb197852db4b20a84aa6c0862468ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:35:23.991637 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-770f57ea-00aa-4c7f-901d-3c7fa5ed42be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:24.201438 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7ecf83-bb84-497e-b986-8034c6a63186 service swift] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:24.201438 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-4f7ecf83-bb84-497e-b986-8034c6a63186 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:24.223917 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7ecf83-bb84-497e-b986-8034c6a63186 service swift] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:24.235761 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41257982-cc2b-4c5e-9c6f-98587559783b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:24.236676 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41257982-cc2b-4c5e-9c6f-98587559783b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:24.236992 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41257982-cc2b-4c5e-9c6f-98587559783b admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:24.237274 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41257982-cc2b-4c5e-9c6f-98587559783b admin admin] PATH_INFO: `/v3/users/92b24ef8f7f44c87a55d1410d3cb07d4` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:24.238558 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41257982-cc2b-4c5e-9c6f-98587559783b admin admin] RBAC: Authorizing `identity:delete_user(user_id=92b24ef8f7f44c87a55d1410d3cb07d4)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:24.240546 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41257982-cc2b-4c5e-9c6f-98587559783b admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:24.246499 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 872/1726] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:35:23 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 321 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:35:24.280788 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770f57ea-00aa-4c7f-901d-3c7fa5ed42be None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:24.284054 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-770f57ea-00aa-4c7f-901d-3c7fa5ed42be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:24.366695 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770f57ea-00aa-4c7f-901d-3c7fa5ed42be None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:24.377840 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b338b6c1-f3e8-4091-ac3e-513c774c78ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:24.379015 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b338b6c1-f3e8-4091-ac3e-513c774c78ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:24.397764 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b338b6c1-f3e8-4091-ac3e-513c774c78ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:24.399533 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b338b6c1-f3e8-4091-ac3e-513c774c78ab admin admin] PATH_INFO: `/v3/projects/77733b7b070540db93d486df83e565f1` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:24.419022 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b338b6c1-f3e8-4091-ac3e-513c774c78ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=77733b7b070540db93d486df83e565f1)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:24.419022 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b338b6c1-f3e8-4091-ac3e-513c774c78ab admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:24.638814 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-41257982-cc2b-4c5e-9c6f-98587559783b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '92b24ef8f7f44c87a55d1410d3cb07d4'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:24.672336 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-b338b6c1-f3e8-4091-ac3e-513c774c78ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77733b7b070540db93d486df83e565f1', 'request_id': 'req-b338b6c1-f3e8-4091-ac3e-513c774c78ab'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:24.715948 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-b338b6c1-f3e8-4091-ac3e-513c774c78ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77733b7b070540db93d486df83e565f1', 'request_id': 'req-b338b6c1-f3e8-4091-ac3e-513c774c78ab'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:24.728782 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:35:24.767788 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-41257982-cc2b-4c5e-9c6f-98587559783b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '92b24ef8f7f44c87a55d1410d3cb07d4'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:24.768658 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-41257982-cc2b-4c5e-9c6f-98587559783b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '92b24ef8f7f44c87a55d1410d3cb07d4'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:24.801791 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-41257982-cc2b-4c5e-9c6f-98587559783b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '92b24ef8f7f44c87a55d1410d3cb07d4'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:24.803005 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-41257982-cc2b-4c5e-9c6f-98587559783b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '92b24ef8f7f44c87a55d1410d3cb07d4'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:24.837404 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 855/1727] 10.222.0.66 () {58 vars in 1223 bytes} [Mon Jul 18 02:35:24 2022] DELETE /identity/v3/users/92b24ef8f7f44c87a55d1410d3cb07d4 => generated 0 bytes in 637 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 02:35:24.842459 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027a1c44-8eb2-4a82-b983-6d06d9af2e53 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:24.844163 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027a1c44-8eb2-4a82-b983-6d06d9af2e53 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:24.844555 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027a1c44-8eb2-4a82-b983-6d06d9af2e53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:24.907047 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-027a1c44-8eb2-4a82-b983-6d06d9af2e53 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:35:24.908471 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-027a1c44-8eb2-4a82-b983-6d06d9af2e53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:24.931709 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-b338b6c1-f3e8-4091-ac3e-513c774c78ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77733b7b070540db93d486df83e565f1', 'request_id': 'req-b338b6c1-f3e8-4091-ac3e-513c774c78ab'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:24.962644 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-b338b6c1-f3e8-4091-ac3e-513c774c78ab admin admin] The token cache is being invalidate because project 77733b7b070540db93d486df83e565f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 02:35:24.965331 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-b338b6c1-f3e8-4091-ac3e-513c774c78ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:24.972713 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-b338b6c1-f3e8-4091-ac3e-513c774c78ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:24.972713 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 873/1728] 10.222.0.66 () {58 vars in 1232 bytes} [Mon Jul 18 02:35:24 2022] DELETE /identity/v3/projects/77733b7b070540db93d486df83e565f1 => generated 0 bytes in 691 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:35:24.987566 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8566234e-d91a-4bf3-a054-d8001878ff91 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:24.988917 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8566234e-d91a-4bf3-a054-d8001878ff91 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:24.990773 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8566234e-d91a-4bf3-a054-d8001878ff91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:25.071981 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 856/1729] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:35:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:35:25.077778 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-8566234e-d91a-4bf3-a054-d8001878ff91 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:35:25.079253 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-8566234e-d91a-4bf3-a054-d8001878ff91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:25.112376 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 874/1730] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:35:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:35:25.112642 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027a1c44-8eb2-4a82-b983-6d06d9af2e53 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:25.112642 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-027a1c44-8eb2-4a82-b983-6d06d9af2e53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:25.119099 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787f8f1d-006b-42be-8558-97aaea5cc5d5 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:25.119674 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787f8f1d-006b-42be-8558-97aaea5cc5d5 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:25.121274 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787f8f1d-006b-42be-8558-97aaea5cc5d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:25.217776 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-787f8f1d-006b-42be-8558-97aaea5cc5d5 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:35:25.219467 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-787f8f1d-006b-42be-8558-97aaea5cc5d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:25.232661 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 875/1731] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:35:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:35:25.233718 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787f8f1d-006b-42be-8558-97aaea5cc5d5 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:25.238588 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-787f8f1d-006b-42be-8558-97aaea5cc5d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:25.253124 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027a1c44-8eb2-4a82-b983-6d06d9af2e53 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:25.264570 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2ec8c6-0851-4532-ac30-5474566ae472 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eaf19a259c2741cea375a38763e33cdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:25.264706 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787f8f1d-006b-42be-8558-97aaea5cc5d5 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:25.265696 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2ec8c6-0851-4532-ac30-5474566ae472 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:25.266141 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2ec8c6-0851-4532-ac30-5474566ae472 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:25.267565 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2ec8c6-0851-4532-ac30-5474566ae472 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:25.272087 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c59eb9-ba99-46e6-b037-753e3625e723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:25.273132 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c59eb9-ba99-46e6-b037-753e3625e723 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:25.273411 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c59eb9-ba99-46e6-b037-753e3625e723 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:25.273692 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c59eb9-ba99-46e6-b037-753e3625e723 admin admin] PATH_INFO: `/v3/users/77989d2930274bd2832378750f807683` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:25.275924 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c59eb9-ba99-46e6-b037-753e3625e723 admin admin] RBAC: Authorizing `identity:delete_user(user_id=77989d2930274bd2832378750f807683)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:25.278815 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c59eb9-ba99-46e6-b037-753e3625e723 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:25.307696 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2ec8c6-0851-4532-ac30-5474566ae472 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:25.314720 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2ec8c6-0851-4532-ac30-5474566ae472 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:25.358766 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 857/1732] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:35:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 02:35:25.362522 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2ec8c6-0851-4532-ac30-5474566ae472 service neutron] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:25.399490 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2ec8c6-0851-4532-ac30-5474566ae472 service neutron] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:25.422052 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5dea68-bbfb-45dc-9474-ef496d89c871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:25.422771 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5dea68-bbfb-45dc-9474-ef496d89c871 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:25.423060 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5dea68-bbfb-45dc-9474-ef496d89c871 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:25.428466 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5dea68-bbfb-45dc-9474-ef496d89c871 admin admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:25.429385 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5dea68-bbfb-45dc-9474-ef496d89c871 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:35:25.429715 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5dea68-bbfb-45dc-9474-ef496d89c871 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:25.431472 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5dea68-bbfb-45dc-9474-ef496d89c871 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:25.502522 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 858/1733] 10.222.0.66 () {58 vars in 1152 bytes} [Mon Jul 18 02:35:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:35:25.504211 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f3da2c-0ed2-4eb6-a748-6333353b80d9 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:25.504522 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f3da2c-0ed2-4eb6-a748-6333353b80d9 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:25.504797 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f3da2c-0ed2-4eb6-a748-6333353b80d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:25.526583 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-b8f3da2c-0ed2-4eb6-a748-6333353b80d9 None None] MFA Rules not processed for user `eaf19a259c2741cea375a38763e33cdc`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:35:25.534403 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-b8f3da2c-0ed2-4eb6-a748-6333353b80d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:25.546510 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 859/1734] 10.222.0.72 () {58 vars in 1015 bytes} [Mon Jul 18 02:35:25 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:35:25.548937 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f3da2c-0ed2-4eb6-a748-6333353b80d9 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:25.570969 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f3da2c-0ed2-4eb6-a748-6333353b80d9 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:25.583958 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639966af-5a87-4c91-9030-bf5f9f1d78c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:25.585991 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639966af-5a87-4c91-9030-bf5f9f1d78c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:25.586360 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639966af-5a87-4c91-9030-bf5f9f1d78c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:25.586565 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-60c59eb9-ba99-46e6-b037-753e3625e723 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77989d2930274bd2832378750f807683'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:25.586675 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639966af-5a87-4c91-9030-bf5f9f1d78c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:25.587330 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639966af-5a87-4c91-9030-bf5f9f1d78c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:25.589351 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639966af-5a87-4c91-9030-bf5f9f1d78c3 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:25.604855 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:35:25.751587 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-60c59eb9-ba99-46e6-b037-753e3625e723 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77989d2930274bd2832378750f807683'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:25.761110 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-60c59eb9-ba99-46e6-b037-753e3625e723 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77989d2930274bd2832378750f807683'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:25.761887 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-60c59eb9-ba99-46e6-b037-753e3625e723 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77989d2930274bd2832378750f807683'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:25.790588 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-60c59eb9-ba99-46e6-b037-753e3625e723 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77989d2930274bd2832378750f807683'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:25.795452 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 860/1735] 10.222.0.66 () {60 vars in 1152 bytes} [Mon Jul 18 02:35:25 2022] POST /identity/v3/projects => generated 363 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:35:25.795452 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639966af-5a87-4c91-9030-bf5f9f1d78c3 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:25.796719 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-639966af-5a87-4c91-9030-bf5f9f1d78c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:25.828935 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 876/1736] 10.222.0.66 () {58 vars in 1223 bytes} [Mon Jul 18 02:35:25 2022] DELETE /identity/v3/users/77989d2930274bd2832378750f807683 => generated 0 bytes in 595 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:35:25.832594 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c59eb9-ba99-46e6-b037-753e3625e723 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:25.841777 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-60c59eb9-ba99-46e6-b037-753e3625e723 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:26.017439 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c59eb9-ba99-46e6-b037-753e3625e723 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:26.028758 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639966af-5a87-4c91-9030-bf5f9f1d78c3 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:26.033161 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ea96f7-1b98-435a-a8d7-093a95091625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:26.034032 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ea96f7-1b98-435a-a8d7-093a95091625 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:26.034806 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ea96f7-1b98-435a-a8d7-093a95091625 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:26.034806 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ea96f7-1b98-435a-a8d7-093a95091625 admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:26.035426 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ea96f7-1b98-435a-a8d7-093a95091625 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:26.038899 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ea96f7-1b98-435a-a8d7-093a95091625 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:26.146865 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51b40cc-6da2-43c0-8c05-c304b8222c70 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eaf19a259c2741cea375a38763e33cdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:26.147677 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51b40cc-6da2-43c0-8c05-c304b8222c70 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:26.163336 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51b40cc-6da2-43c0-8c05-c304b8222c70 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:26.167659 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51b40cc-6da2-43c0-8c05-c304b8222c70 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:26.167957 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 877/1737] 10.222.0.66 () {60 vars in 1143 bytes} [Mon Jul 18 02:35:25 2022] POST /identity/v3/users => generated 282 bytes in 337 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:35:26.168446 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315c0f48-d17e-4d17-ad9b-485e39105ccd None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:26.168741 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315c0f48-d17e-4d17-ad9b-485e39105ccd None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:26.169604 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315c0f48-d17e-4d17-ad9b-485e39105ccd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:26.175838 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-d51b40cc-6da2-43c0-8c05-c304b8222c70 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:26.204320 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51b40cc-6da2-43c0-8c05-c304b8222c70 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:26.205885 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51b40cc-6da2-43c0-8c05-c304b8222c70 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:26.229499 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-315c0f48-d17e-4d17-ad9b-485e39105ccd None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:35:26.234042 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-315c0f48-d17e-4d17-ad9b-485e39105ccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:26.242311 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 861/1738] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:35:25 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:35:26.245057 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51b40cc-6da2-43c0-8c05-c304b8222c70 service neutron] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:26.283751 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 878/1739] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:35:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:35:26.309003 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51b40cc-6da2-43c0-8c05-c304b8222c70 service neutron] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:26.328929 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315c0f48-d17e-4d17-ad9b-485e39105ccd None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:26.365069 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c81be2a-93be-49ae-b66c-00d6b7b63ba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:26.371420 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c81be2a-93be-49ae-b66c-00d6b7b63ba6 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:26.371722 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c81be2a-93be-49ae-b66c-00d6b7b63ba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:26.372053 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c81be2a-93be-49ae-b66c-00d6b7b63ba6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:26.377831 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315c0f48-d17e-4d17-ad9b-485e39105ccd None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:26.384325 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c81be2a-93be-49ae-b66c-00d6b7b63ba6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:26.385928 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c81be2a-93be-49ae-b66c-00d6b7b63ba6 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:26.405202 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcd7db6-72b7-42a2-bd6f-b61c2d37d0a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eaf19a259c2741cea375a38763e33cdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:26.411042 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcd7db6-72b7-42a2-bd6f-b61c2d37d0a5 service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:26.411965 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcd7db6-72b7-42a2-bd6f-b61c2d37d0a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:26.411965 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcd7db6-72b7-42a2-bd6f-b61c2d37d0a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:26.436499 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 862/1740] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:35:26 2022] GET /identity/v3/roles => generated 1420 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:35:26.451773 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c81be2a-93be-49ae-b66c-00d6b7b63ba6 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:26.483151 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c81be2a-93be-49ae-b66c-00d6b7b63ba6 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:26.490000 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcd7db6-72b7-42a2-bd6f-b61c2d37d0a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:26.501742 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e040caf8-4860-4bb5-9334-ba2e1db68540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:26.504546 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e040caf8-4860-4bb5-9334-ba2e1db68540 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:26.505132 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e040caf8-4860-4bb5-9334-ba2e1db68540 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:26.510507 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcd7db6-72b7-42a2-bd6f-b61c2d37d0a5 service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:26.544158 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e040caf8-4860-4bb5-9334-ba2e1db68540 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:26.545059 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e040caf8-4860-4bb5-9334-ba2e1db68540 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:26.554408 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e040caf8-4860-4bb5-9334-ba2e1db68540 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:26.579847 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 879/1741] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:35:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:35:26.612224 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 863/1742] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:35:26 2022] GET /identity/v3/roles => generated 1420 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:35:26.612738 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcd7db6-72b7-42a2-bd6f-b61c2d37d0a5 service neutron] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:26.630834 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcd7db6-72b7-42a2-bd6f-b61c2d37d0a5 service neutron] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:26.634338 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e040caf8-4860-4bb5-9334-ba2e1db68540 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:26.655328 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-e040caf8-4860-4bb5-9334-ba2e1db68540 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:26.665296 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796fb200-d8f4-4c74-b141-f6204457c7bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:26.668173 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796fb200-d8f4-4c74-b141-f6204457c7bd admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:26.670278 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796fb200-d8f4-4c74-b141-f6204457c7bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:26.673443 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796fb200-d8f4-4c74-b141-f6204457c7bd admin admin] PATH_INFO: `/v3/projects/9179e44bf9d74d50ae488bdeca0723df/users/abd6b032a11047a88019d785e60b715a/roles/65feabf3b27944a8b01a86c8ac608457` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:26.710339 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e040caf8-4860-4bb5-9334-ba2e1db68540 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:26.730691 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bd7b6f-b924-468d-9bff-1312ea047041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:26.732088 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bd7b6f-b924-468d-9bff-1312ea047041 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:26.732561 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bd7b6f-b924-468d-9bff-1312ea047041 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:26.733024 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bd7b6f-b924-468d-9bff-1312ea047041 admin admin] PATH_INFO: `/v3/projects/111367517d7f4746b244db54a7af290a` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:26.736978 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bd7b6f-b924-468d-9bff-1312ea047041 admin admin] RBAC: Authorizing `identity:delete_project(project_id=111367517d7f4746b244db54a7af290a)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:26.739237 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bd7b6f-b924-468d-9bff-1312ea047041 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:26.746340 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:35:26.752063 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796fb200-d8f4-4c74-b141-f6204457c7bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=9179e44bf9d74d50ae488bdeca0723df, user_id=abd6b032a11047a88019d785e60b715a, role_id=65feabf3b27944a8b01a86c8ac608457)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:26.760703 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796fb200-d8f4-4c74-b141-f6204457c7bd admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:27.055957 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 880/1743] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:35:26 2022] PUT /identity/v3/projects/9179e44bf9d74d50ae488bdeca0723df/users/abd6b032a11047a88019d785e60b715a/roles/65feabf3b27944a8b01a86c8ac608457 => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 02:35:27.067313 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb02379f-ab95-430c-b7c7-c0687e3f73a5 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:27.068162 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb02379f-ab95-430c-b7c7-c0687e3f73a5 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:27.068761 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb02379f-ab95-430c-b7c7-c0687e3f73a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:27.102713 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-11bd7b6f-b924-468d-9bff-1312ea047041 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '111367517d7f4746b244db54a7af290a', 'request_id': 'req-11bd7b6f-b924-468d-9bff-1312ea047041'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:27.117056 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-11bd7b6f-b924-468d-9bff-1312ea047041 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '111367517d7f4746b244db54a7af290a', 'request_id': 'req-11bd7b6f-b924-468d-9bff-1312ea047041'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:27.131857 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-11bd7b6f-b924-468d-9bff-1312ea047041 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '111367517d7f4746b244db54a7af290a', 'request_id': 'req-11bd7b6f-b924-468d-9bff-1312ea047041'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:27.136567 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-eb02379f-ab95-430c-b7c7-c0687e3f73a5 None None] MFA Rules not processed for user `abd6b032a11047a88019d785e60b715a`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:35:27.137968 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-eb02379f-ab95-430c-b7c7-c0687e3f73a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:27.266169 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-11bd7b6f-b924-468d-9bff-1312ea047041 admin admin] The token cache is being invalidate because project 111367517d7f4746b244db54a7af290a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 02:35:27.267303 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-11bd7b6f-b924-468d-9bff-1312ea047041 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:27.268527 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-11bd7b6f-b924-468d-9bff-1312ea047041 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:27.271113 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 864/1744] 10.222.0.66 () {58 vars in 1232 bytes} [Mon Jul 18 02:35:26 2022] DELETE /identity/v3/projects/111367517d7f4746b244db54a7af290a => generated 0 bytes in 638 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:35:27.276570 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbcae99-5bd2-496e-8167-fd8e334795d1 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:27.277556 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbcae99-5bd2-496e-8167-fd8e334795d1 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:27.278261 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbcae99-5bd2-496e-8167-fd8e334795d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:27.341781 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 881/1745] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:35:27 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 02:35:27.350550 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb02379f-ab95-430c-b7c7-c0687e3f73a5 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:27.353975 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-9fbcae99-5bd2-496e-8167-fd8e334795d1 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:35:27.355559 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-9fbcae99-5bd2-496e-8167-fd8e334795d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:27.358477 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-eb02379f-ab95-430c-b7c7-c0687e3f73a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:27.501130 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb02379f-ab95-430c-b7c7-c0687e3f73a5 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:27.527425 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac30a82b-845e-4e3a-be55-2fe5fbd462de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:27.528475 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac30a82b-845e-4e3a-be55-2fe5fbd462de admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:27.528947 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac30a82b-845e-4e3a-be55-2fe5fbd462de admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:27.529426 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac30a82b-845e-4e3a-be55-2fe5fbd462de admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:27.530663 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac30a82b-845e-4e3a-be55-2fe5fbd462de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:27.535135 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac30a82b-845e-4e3a-be55-2fe5fbd462de admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:27.639062 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 865/1746] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:35:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 358 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:35:27.642253 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9a402a-60b0-4fb2-9b45-29462adc9acc None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:27.642253 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9a402a-60b0-4fb2-9b45-29462adc9acc None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:27.642253 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9a402a-60b0-4fb2-9b45-29462adc9acc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:27.720711 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 882/1747] 10.222.0.66 () {60 vars in 1152 bytes} [Mon Jul 18 02:35:27 2022] POST /identity/v3/projects => generated 365 bytes in 370 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:35:27.721248 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-5b9a402a-60b0-4fb2-9b45-29462adc9acc None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:35:27.721863 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac30a82b-845e-4e3a-be55-2fe5fbd462de admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:27.743603 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-5b9a402a-60b0-4fb2-9b45-29462adc9acc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:27.939557 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 866/1748] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:35:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:35:27.945682 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9a402a-60b0-4fb2-9b45-29462adc9acc None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:27.947763 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-5b9a402a-60b0-4fb2-9b45-29462adc9acc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:27.973263 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9a402a-60b0-4fb2-9b45-29462adc9acc None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:27.980226 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea97b1b4-ba07-41d5-8568-8b19e7b0136a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:27.985809 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea97b1b4-ba07-41d5-8568-8b19e7b0136a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:27.985809 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea97b1b4-ba07-41d5-8568-8b19e7b0136a admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:27.985809 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea97b1b4-ba07-41d5-8568-8b19e7b0136a admin admin] PATH_INFO: `/v3/users/253d2bf6c1aa42e6a55a760e5e37f01a` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:27.985809 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea97b1b4-ba07-41d5-8568-8b19e7b0136a admin admin] RBAC: Authorizing `identity:delete_user(user_id=253d2bf6c1aa42e6a55a760e5e37f01a)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:27.987777 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea97b1b4-ba07-41d5-8568-8b19e7b0136a admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:28.015080 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac30a82b-845e-4e3a-be55-2fe5fbd462de admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:28.039047 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0e8f46-d13c-4fb8-9c39-1429704bc110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:28.047215 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0e8f46-d13c-4fb8-9c39-1429704bc110 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:28.047215 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0e8f46-d13c-4fb8-9c39-1429704bc110 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:28.047215 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0e8f46-d13c-4fb8-9c39-1429704bc110 admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:28.049430 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0e8f46-d13c-4fb8-9c39-1429704bc110 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:28.054617 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0e8f46-d13c-4fb8-9c39-1429704bc110 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:28.268796 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 883/1749] 10.222.0.66 () {60 vars in 1143 bytes} [Mon Jul 18 02:35:27 2022] POST /identity/v3/users => generated 289 bytes in 548 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:35:28.272792 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0e8f46-d13c-4fb8-9c39-1429704bc110 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:28.284593 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0e8f46-d13c-4fb8-9c39-1429704bc110 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:28.291077 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e061130-bb3a-4fe0-b26e-42bb4f5cee3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:28.292034 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e061130-bb3a-4fe0-b26e-42bb4f5cee3b admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:28.292541 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e061130-bb3a-4fe0-b26e-42bb4f5cee3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:28.292995 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e061130-bb3a-4fe0-b26e-42bb4f5cee3b admin admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:28.294586 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e061130-bb3a-4fe0-b26e-42bb4f5cee3b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:35:28.295200 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e061130-bb3a-4fe0-b26e-42bb4f5cee3b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:28.297095 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e061130-bb3a-4fe0-b26e-42bb4f5cee3b admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:28.316371 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 884/1750] 10.222.0.66 () {58 vars in 1152 bytes} [Mon Jul 18 02:35:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:35:28.317387 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e061130-bb3a-4fe0-b26e-42bb4f5cee3b admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:28.328875 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e061130-bb3a-4fe0-b26e-42bb4f5cee3b admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:28.338507 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3162e2d-e187-4621-9399-01fc29be86ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:28.347145 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3162e2d-e187-4621-9399-01fc29be86ce admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:28.347145 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3162e2d-e187-4621-9399-01fc29be86ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:28.347145 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3162e2d-e187-4621-9399-01fc29be86ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:28.347145 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3162e2d-e187-4621-9399-01fc29be86ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:28.347145 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3162e2d-e187-4621-9399-01fc29be86ce admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:28.374385 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 885/1751] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:35:28 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:35:28.377668 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3162e2d-e187-4621-9399-01fc29be86ce admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:28.392196 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-ea97b1b4-ba07-41d5-8568-8b19e7b0136a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '253d2bf6c1aa42e6a55a760e5e37f01a'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:28.438949 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3162e2d-e187-4621-9399-01fc29be86ce admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:28.448073 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60809705-be5e-4597-9ec5-a0bfe2970e23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:28.449252 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60809705-be5e-4597-9ec5-a0bfe2970e23 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:28.449713 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60809705-be5e-4597-9ec5-a0bfe2970e23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:28.451359 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60809705-be5e-4597-9ec5-a0bfe2970e23 admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:28.452480 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60809705-be5e-4597-9ec5-a0bfe2970e23 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:28.457136 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60809705-be5e-4597-9ec5-a0bfe2970e23 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:28.541737 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-ea97b1b4-ba07-41d5-8568-8b19e7b0136a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '253d2bf6c1aa42e6a55a760e5e37f01a'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:28.545320 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-ea97b1b4-ba07-41d5-8568-8b19e7b0136a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '253d2bf6c1aa42e6a55a760e5e37f01a'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:28.578045 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-ea97b1b4-ba07-41d5-8568-8b19e7b0136a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '253d2bf6c1aa42e6a55a760e5e37f01a'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:28.579175 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-ea97b1b4-ba07-41d5-8568-8b19e7b0136a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '253d2bf6c1aa42e6a55a760e5e37f01a'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:28.593016 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:35:28.642868 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 867/1752] 10.222.0.66 () {58 vars in 1223 bytes} [Mon Jul 18 02:35:27 2022] DELETE /identity/v3/users/253d2bf6c1aa42e6a55a760e5e37f01a => generated 0 bytes in 699 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:35:28.646869 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 886/1753] 10.222.0.66 () {60 vars in 1152 bytes} [Mon Jul 18 02:35:28 2022] POST /identity/v3/projects => generated 403 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:35:28.649016 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea97b1b4-ba07-41d5-8568-8b19e7b0136a admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:28.650884 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60809705-be5e-4597-9ec5-a0bfe2970e23 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:28.653402 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:35:28.657392 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-60809705-be5e-4597-9ec5-a0bfe2970e23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:28.657797 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-ea97b1b4-ba07-41d5-8568-8b19e7b0136a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:28.778986 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea97b1b4-ba07-41d5-8568-8b19e7b0136a admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:28.932332 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a5eae5-e574-4ea0-ba27-e014e6f3d88f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:28.938533 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a5eae5-e574-4ea0-ba27-e014e6f3d88f admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:28.938533 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a5eae5-e574-4ea0-ba27-e014e6f3d88f admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:28.938533 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a5eae5-e574-4ea0-ba27-e014e6f3d88f admin admin] PATH_INFO: `/v3/projects/4d08f76a482c48daa325fa385c2805c4/users/41c43858514f4d82bf826a58177b5eb8/roles/d4aa3324f0c84887b41d0dc4acf7f30e` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:28.939004 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60809705-be5e-4597-9ec5-a0bfe2970e23 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:28.980325 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626c7a9a-51f3-4217-b227-a04d7ab1d4f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:28.982977 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a5eae5-e574-4ea0-ba27-e014e6f3d88f admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d08f76a482c48daa325fa385c2805c4, user_id=41c43858514f4d82bf826a58177b5eb8, role_id=d4aa3324f0c84887b41d0dc4acf7f30e)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:28.985696 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a5eae5-e574-4ea0-ba27-e014e6f3d88f admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:28.992454 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626c7a9a-51f3-4217-b227-a04d7ab1d4f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:28.992828 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626c7a9a-51f3-4217-b227-a04d7ab1d4f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:28.994724 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626c7a9a-51f3-4217-b227-a04d7ab1d4f8 admin admin] PATH_INFO: `/v3/projects/9917e65b8ee449038cbd72443c368db7` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:28.997516 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626c7a9a-51f3-4217-b227-a04d7ab1d4f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9917e65b8ee449038cbd72443c368db7)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:28.999521 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626c7a9a-51f3-4217-b227-a04d7ab1d4f8 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:29.109718 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 868/1754] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:35:28 2022] PUT /identity/v3/projects/4d08f76a482c48daa325fa385c2805c4/users/41c43858514f4d82bf826a58177b5eb8/roles/d4aa3324f0c84887b41d0dc4acf7f30e => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:35:29.113734 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873305cc-1bb4-412a-950b-3fd6f1396277 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:29.113734 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873305cc-1bb4-412a-950b-3fd6f1396277 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:29.113734 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873305cc-1bb4-412a-950b-3fd6f1396277 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:29.195736 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-873305cc-1bb4-412a-950b-3fd6f1396277 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:35:29.203809 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-873305cc-1bb4-412a-950b-3fd6f1396277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:29.254047 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-626c7a9a-51f3-4217-b227-a04d7ab1d4f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9917e65b8ee449038cbd72443c368db7', 'request_id': 'req-626c7a9a-51f3-4217-b227-a04d7ab1d4f8'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:29.269475 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-626c7a9a-51f3-4217-b227-a04d7ab1d4f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9917e65b8ee449038cbd72443c368db7', 'request_id': 'req-626c7a9a-51f3-4217-b227-a04d7ab1d4f8'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:29.437561 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 869/1755] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:35:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 327 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:35:29.439271 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-626c7a9a-51f3-4217-b227-a04d7ab1d4f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9917e65b8ee449038cbd72443c368db7', 'request_id': 'req-626c7a9a-51f3-4217-b227-a04d7ab1d4f8'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:29.440630 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873305cc-1bb4-412a-950b-3fd6f1396277 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:29.442702 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-873305cc-1bb4-412a-950b-3fd6f1396277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:29.475140 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873305cc-1bb4-412a-950b-3fd6f1396277 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:29.478948 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-626c7a9a-51f3-4217-b227-a04d7ab1d4f8 admin admin] The token cache is being invalidate because project 9917e65b8ee449038cbd72443c368db7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 02:35:29.482438 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-626c7a9a-51f3-4217-b227-a04d7ab1d4f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:29.484730 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-626c7a9a-51f3-4217-b227-a04d7ab1d4f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:29.491865 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 887/1756] 10.222.0.66 () {58 vars in 1232 bytes} [Mon Jul 18 02:35:28 2022] DELETE /identity/v3/projects/9917e65b8ee449038cbd72443c368db7 => generated 0 bytes in 844 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:35:29.493044 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626c7a9a-51f3-4217-b227-a04d7ab1d4f8 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:29.507016 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-626c7a9a-51f3-4217-b227-a04d7ab1d4f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:29.529935 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c83ce5-0d91-4df3-ab03-2eb4448c180f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:29.532130 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c83ce5-0d91-4df3-ab03-2eb4448c180f admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:29.536071 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c83ce5-0d91-4df3-ab03-2eb4448c180f admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:29.536071 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c83ce5-0d91-4df3-ab03-2eb4448c180f admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:29.536071 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c83ce5-0d91-4df3-ab03-2eb4448c180f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:29.538639 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c83ce5-0d91-4df3-ab03-2eb4448c180f admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:29.551240 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626c7a9a-51f3-4217-b227-a04d7ab1d4f8 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:29.558858 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c45b1c-fbb4-4f6d-8ef5-7422ae6bcd31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:29.561833 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c45b1c-fbb4-4f6d-8ef5-7422ae6bcd31 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:29.562354 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c45b1c-fbb4-4f6d-8ef5-7422ae6bcd31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:29.565706 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c45b1c-fbb4-4f6d-8ef5-7422ae6bcd31 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:29.565706 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c45b1c-fbb4-4f6d-8ef5-7422ae6bcd31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:29.565706 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c45b1c-fbb4-4f6d-8ef5-7422ae6bcd31 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:29.582834 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 888/1757] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:35:29 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:35:29.584618 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd41baf9-a138-4d5d-95a7-68e9a5abbed5 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:29.584908 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd41baf9-a138-4d5d-95a7-68e9a5abbed5 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:29.585279 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd41baf9-a138-4d5d-95a7-68e9a5abbed5 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:29.586996 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 889/1758] 10.222.0.66 () {54 vars in 921 bytes} [Mon Jul 18 02:35:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:35:29.594544 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd41baf9-a138-4d5d-95a7-68e9a5abbed5 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:29.596431 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-bd41baf9-a138-4d5d-95a7-68e9a5abbed5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:29.679028 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd41baf9-a138-4d5d-95a7-68e9a5abbed5 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:29.685330 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006f3840-2667-47d5-a14e-e57339807082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:29.699166 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006f3840-2667-47d5-a14e-e57339807082 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:29.699166 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006f3840-2667-47d5-a14e-e57339807082 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:29.699166 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006f3840-2667-47d5-a14e-e57339807082 admin admin] PATH_INFO: `/v3/users/87fcd3f1c4fa41fe94a59def1e79541e` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:29.699166 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006f3840-2667-47d5-a14e-e57339807082 admin admin] RBAC: Authorizing `identity:delete_user(user_id=87fcd3f1c4fa41fe94a59def1e79541e)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:29.715144 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006f3840-2667-47d5-a14e-e57339807082 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:29.721860 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 870/1759] 10.222.0.66 () {60 vars in 1143 bytes} [Mon Jul 18 02:35:29 2022] POST /identity/v3/users => generated 302 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:35:29.723005 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c83ce5-0d91-4df3-ab03-2eb4448c180f admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:29.726436 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-48c83ce5-0d91-4df3-ab03-2eb4448c180f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:29.853398 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c83ce5-0d91-4df3-ab03-2eb4448c180f admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:29.907078 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677db621-64e0-4733-9f23-109396a6a674 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eaf19a259c2741cea375a38763e33cdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:29.907924 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677db621-64e0-4733-9f23-109396a6a674 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:29.908286 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677db621-64e0-4733-9f23-109396a6a674 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:29.908636 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677db621-64e0-4733-9f23-109396a6a674 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:29.914737 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-677db621-64e0-4733-9f23-109396a6a674 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:29.944239 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677db621-64e0-4733-9f23-109396a6a674 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:29.946699 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677db621-64e0-4733-9f23-109396a6a674 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:29.987469 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 871/1760] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:35:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:35:29.994121 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677db621-64e0-4733-9f23-109396a6a674 service neutron] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:30.036212 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677db621-64e0-4733-9f23-109396a6a674 service neutron] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:30.057197 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7e2de4-75d0-465d-b238-aedf61a77686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:30.058354 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7e2de4-75d0-465d-b238-aedf61a77686 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:30.058490 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7e2de4-75d0-465d-b238-aedf61a77686 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:30.058748 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7e2de4-75d0-465d-b238-aedf61a77686 admin admin] PATH_INFO: `/v3/projects/4d08f76a482c48daa325fa385c2805c4/users/41c43858514f4d82bf826a58177b5eb8/roles/d4aa3324f0c84887b41d0dc4acf7f30e` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:30.064525 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7e2de4-75d0-465d-b238-aedf61a77686 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d08f76a482c48daa325fa385c2805c4, user_id=41c43858514f4d82bf826a58177b5eb8, role_id=d4aa3324f0c84887b41d0dc4acf7f30e)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:30.067353 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7e2de4-75d0-465d-b238-aedf61a77686 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:30.075317 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-006f3840-2667-47d5-a14e-e57339807082 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87fcd3f1c4fa41fe94a59def1e79541e'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:30.102832 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 872/1761] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:35:29 2022] PUT /identity/v3/projects/4d08f76a482c48daa325fa385c2805c4/users/41c43858514f4d82bf826a58177b5eb8/roles/d4aa3324f0c84887b41d0dc4acf7f30e => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:35:30.117730 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185a949b-6783-4cef-a9d7-44f9ca29b4d8 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:30.117730 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185a949b-6783-4cef-a9d7-44f9ca29b4d8 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:30.117730 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185a949b-6783-4cef-a9d7-44f9ca29b4d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:30.174381 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-185a949b-6783-4cef-a9d7-44f9ca29b4d8 None None] MFA Rules not processed for user `80b138831bc54c7b986d50092d62d4af`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:35:30.184874 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-185a949b-6783-4cef-a9d7-44f9ca29b4d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:30.189276 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-006f3840-2667-47d5-a14e-e57339807082 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87fcd3f1c4fa41fe94a59def1e79541e'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:30.199464 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-006f3840-2667-47d5-a14e-e57339807082 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87fcd3f1c4fa41fe94a59def1e79541e'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:30.200812 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-006f3840-2667-47d5-a14e-e57339807082 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87fcd3f1c4fa41fe94a59def1e79541e'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:30.231702 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-006f3840-2667-47d5-a14e-e57339807082 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87fcd3f1c4fa41fe94a59def1e79541e'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:30.244276 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 890/1762] 10.222.0.66 () {58 vars in 1223 bytes} [Mon Jul 18 02:35:29 2022] DELETE /identity/v3/users/87fcd3f1c4fa41fe94a59def1e79541e => generated 0 bytes in 654 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:35:30.244276 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006f3840-2667-47d5-a14e-e57339807082 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:30.244704 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-006f3840-2667-47d5-a14e-e57339807082 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:30.355604 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 873/1763] 10.222.0.66 () {58 vars in 1018 bytes} [Mon Jul 18 02:35:30 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:35:30.362007 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afe68dd-c40c-4bc7-ae8c-4e470d550a26 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:30.364865 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afe68dd-c40c-4bc7-ae8c-4e470d550a26 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:30.364865 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afe68dd-c40c-4bc7-ae8c-4e470d550a26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:30.430644 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006f3840-2667-47d5-a14e-e57339807082 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:30.430644 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee70604-dd17-4d42-be9c-99c0fbbf7ce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:30.430644 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee70604-dd17-4d42-be9c-99c0fbbf7ce9 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:30.430644 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee70604-dd17-4d42-be9c-99c0fbbf7ce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:30.431454 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee70604-dd17-4d42-be9c-99c0fbbf7ce9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:30.431454 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee70604-dd17-4d42-be9c-99c0fbbf7ce9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:30.431454 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee70604-dd17-4d42-be9c-99c0fbbf7ce9 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:30.432490 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-8afe68dd-c40c-4bc7-ae8c-4e470d550a26 None None] MFA Rules not processed for user `41c43858514f4d82bf826a58177b5eb8`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:35:30.433977 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-8afe68dd-c40c-4bc7-ae8c-4e470d550a26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:30.476656 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:35:30.489376 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 891/1764] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:35:30 2022] GET /identity/v3/roles => generated 1420 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:35:30.493263 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee70604-dd17-4d42-be9c-99c0fbbf7ce9 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:30.503225 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-cee70604-dd17-4d42-be9c-99c0fbbf7ce9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:30.560191 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:35:30.651877 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 874/1765] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:35:30 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:35:30.652799 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afe68dd-c40c-4bc7-ae8c-4e470d550a26 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:30.688968 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afe68dd-c40c-4bc7-ae8c-4e470d550a26 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:30.695218 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26bd625-b921-4172-87dd-421313ee131f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:30.696236 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26bd625-b921-4172-87dd-421313ee131f admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:30.696628 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26bd625-b921-4172-87dd-421313ee131f admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:30.696877 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26bd625-b921-4172-87dd-421313ee131f admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:30.697913 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26bd625-b921-4172-87dd-421313ee131f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:30.700198 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26bd625-b921-4172-87dd-421313ee131f admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:30.703653 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee70604-dd17-4d42-be9c-99c0fbbf7ce9 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:30.714409 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cdf899-5e7b-4037-a041-6349ccc39c0a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80b138831bc54c7b986d50092d62d4af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:30.715083 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cdf899-5e7b-4037-a041-6349ccc39c0a service nova] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:30.715396 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cdf899-5e7b-4037-a041-6349ccc39c0a service nova] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:30.716167 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cdf899-5e7b-4037-a041-6349ccc39c0a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:30.731720 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 875/1766] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:35:30 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:35:30.732723 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26bd625-b921-4172-87dd-421313ee131f admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:30.739740 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cdf899-5e7b-4037-a041-6349ccc39c0a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:30.739954 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-d26bd625-b921-4172-87dd-421313ee131f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:30.741583 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cdf899-5e7b-4037-a041-6349ccc39c0a service nova] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:30.780325 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 892/1767] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:35:30 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:35:30.782952 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cdf899-5e7b-4037-a041-6349ccc39c0a service nova] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:30.810559 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cdf899-5e7b-4037-a041-6349ccc39c0a service nova] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:30.821200 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8eed421-2b42-4fe8-9ffe-5e9c5382cf33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:30.822811 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eed421-2b42-4fe8-9ffe-5e9c5382cf33 admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:30.823256 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eed421-2b42-4fe8-9ffe-5e9c5382cf33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:30.823725 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eed421-2b42-4fe8-9ffe-5e9c5382cf33 admin admin] PATH_INFO: `/v3/projects/4a7cc87017b94cc68c7ae0304dadadf4/users/234b4142711d4304ba355c4ed0192aa5/roles/65feabf3b27944a8b01a86c8ac608457` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:30.853147 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26bd625-b921-4172-87dd-421313ee131f admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:30.859288 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ab9063-a620-4a1c-b27b-6b623ecbcecb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd49388facdf446885e55b45ef91f01e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:30.859973 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ab9063-a620-4a1c-b27b-6b623ecbcecb service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:30.860266 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ab9063-a620-4a1c-b27b-6b623ecbcecb service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:30.860709 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ab9063-a620-4a1c-b27b-6b623ecbcecb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:30.869801 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8eed421-2b42-4fe8-9ffe-5e9c5382cf33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a7cc87017b94cc68c7ae0304dadadf4, user_id=234b4142711d4304ba355c4ed0192aa5, role_id=65feabf3b27944a8b01a86c8ac608457)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:30.875867 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8eed421-2b42-4fe8-9ffe-5e9c5382cf33 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:30.890702 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ab9063-a620-4a1c-b27b-6b623ecbcecb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:30.895493 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ab9063-a620-4a1c-b27b-6b623ecbcecb service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:30.935242 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 876/1768] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:35:30 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:35:31.030856 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 893/1769] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:35:30 2022] PUT /identity/v3/projects/4a7cc87017b94cc68c7ae0304dadadf4/users/234b4142711d4304ba355c4ed0192aa5/roles/65feabf3b27944a8b01a86c8ac608457 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:35:31.032930 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac172444-b206-4c30-a13d-4e1205b2de31 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:31.033452 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac172444-b206-4c30-a13d-4e1205b2de31 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:31.034635 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac172444-b206-4c30-a13d-4e1205b2de31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:31.097511 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-ac172444-b206-4c30-a13d-4e1205b2de31 None None] MFA Rules not processed for user `234b4142711d4304ba355c4ed0192aa5`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:35:31.100615 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-ac172444-b206-4c30-a13d-4e1205b2de31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:31.241608 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8eed421-2b42-4fe8-9ffe-5e9c5382cf33 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:31.247571 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 877/1770] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:35:31 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:35:31.251199 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac172444-b206-4c30-a13d-4e1205b2de31 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:31.419823 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8eed421-2b42-4fe8-9ffe-5e9c5382cf33 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:31.433069 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee02e94-c337-4ec6-8342-052e2d1821dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd49388facdf446885e55b45ef91f01e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:31.436464 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee02e94-c337-4ec6-8342-052e2d1821dd service cinder] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:31.439612 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee02e94-c337-4ec6-8342-052e2d1821dd service cinder] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:31.440240 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee02e94-c337-4ec6-8342-052e2d1821dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:31.448508 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-0ee02e94-c337-4ec6-8342-052e2d1821dd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:31.485319 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac172444-b206-4c30-a13d-4e1205b2de31 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:31.504251 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b86580-e93b-48a4-a4f2-10931d1b1339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:31.506321 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b86580-e93b-48a4-a4f2-10931d1b1339 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:31.506768 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b86580-e93b-48a4-a4f2-10931d1b1339 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:31.507956 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b86580-e93b-48a4-a4f2-10931d1b1339 admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:31.509035 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b86580-e93b-48a4-a4f2-10931d1b1339 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:31.512599 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b86580-e93b-48a4-a4f2-10931d1b1339 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:31.698856 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee02e94-c337-4ec6-8342-052e2d1821dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:31.718110 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee02e94-c337-4ec6-8342-052e2d1821dd service cinder] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:31.752587 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 894/1771] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:35:31 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 511 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 02:35:31.753402 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee02e94-c337-4ec6-8342-052e2d1821dd service cinder] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:31.754877 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-0ee02e94-c337-4ec6-8342-052e2d1821dd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:31.788968 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee02e94-c337-4ec6-8342-052e2d1821dd service cinder] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:31.801324 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a33c0b-5b72-480e-960f-62362fd446b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:31.802354 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a33c0b-5b72-480e-960f-62362fd446b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:31.803043 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a33c0b-5b72-480e-960f-62362fd446b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:31.806678 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a33c0b-5b72-480e-960f-62362fd446b5 admin admin] PATH_INFO: `/v3/projects/805769e9d58340d48402101bf7eb967e` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:31.811519 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a33c0b-5b72-480e-960f-62362fd446b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=805769e9d58340d48402101bf7eb967e)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:31.813709 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a33c0b-5b72-480e-960f-62362fd446b5 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:31.913201 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 878/1772] 10.222.0.66 () {60 vars in 1152 bytes} [Mon Jul 18 02:35:31 2022] POST /identity/v3/projects => generated 405 bytes in 660 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 02:35:31.920694 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b86580-e93b-48a4-a4f2-10931d1b1339 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:31.936496 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-06b86580-e93b-48a4-a4f2-10931d1b1339 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:31.969594 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-23a33c0b-5b72-480e-960f-62362fd446b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '805769e9d58340d48402101bf7eb967e', 'request_id': 'req-23a33c0b-5b72-480e-960f-62362fd446b5'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:31.982792 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-23a33c0b-5b72-480e-960f-62362fd446b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '805769e9d58340d48402101bf7eb967e', 'request_id': 'req-23a33c0b-5b72-480e-960f-62362fd446b5'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:32.205717 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b86580-e93b-48a4-a4f2-10931d1b1339 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:32.214837 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a68abf-8acb-4bd2-9371-7651a4f9126c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:32.216708 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a68abf-8acb-4bd2-9371-7651a4f9126c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:32.216996 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a68abf-8acb-4bd2-9371-7651a4f9126c admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:32.217277 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a68abf-8acb-4bd2-9371-7651a4f9126c admin admin] PATH_INFO: `/v3/projects/a1adfe26f31649cab9688e5c766ed9da` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:32.219949 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a68abf-8acb-4bd2-9371-7651a4f9126c admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1adfe26f31649cab9688e5c766ed9da)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:32.227891 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a68abf-8acb-4bd2-9371-7651a4f9126c admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:32.255331 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:35:32.276115 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-23a33c0b-5b72-480e-960f-62362fd446b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '805769e9d58340d48402101bf7eb967e', 'request_id': 'req-23a33c0b-5b72-480e-960f-62362fd446b5'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:32.311771 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-23a33c0b-5b72-480e-960f-62362fd446b5 admin admin] The token cache is being invalidate because project 805769e9d58340d48402101bf7eb967e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 02:35:32.312339 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-23a33c0b-5b72-480e-960f-62362fd446b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:32.315129 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-23a33c0b-5b72-480e-960f-62362fd446b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:32.323680 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 895/1773] 10.222.0.66 () {58 vars in 1232 bytes} [Mon Jul 18 02:35:31 2022] DELETE /identity/v3/projects/805769e9d58340d48402101bf7eb967e => generated 0 bytes in 571 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:35:32.329974 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a33c0b-5b72-480e-960f-62362fd446b5 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:32.332793 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-23a33c0b-5b72-480e-960f-62362fd446b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:32.369309 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a33c0b-5b72-480e-960f-62362fd446b5 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:32.376447 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b98754-5565-4814-aa4b-4d46992acfcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:32.377487 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b98754-5565-4814-aa4b-4d46992acfcf admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:32.377897 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b98754-5565-4814-aa4b-4d46992acfcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:32.378277 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b98754-5565-4814-aa4b-4d46992acfcf admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:32.384987 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b98754-5565-4814-aa4b-4d46992acfcf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:32.391254 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b98754-5565-4814-aa4b-4d46992acfcf admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:32.441908 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-55a68abf-8acb-4bd2-9371-7651a4f9126c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1adfe26f31649cab9688e5c766ed9da', 'request_id': 'req-55a68abf-8acb-4bd2-9371-7651a4f9126c'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:32.478479 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-55a68abf-8acb-4bd2-9371-7651a4f9126c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1adfe26f31649cab9688e5c766ed9da', 'request_id': 'req-55a68abf-8acb-4bd2-9371-7651a4f9126c'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:32.500962 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-55a68abf-8acb-4bd2-9371-7651a4f9126c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1adfe26f31649cab9688e5c766ed9da', 'request_id': 'req-55a68abf-8acb-4bd2-9371-7651a4f9126c'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:32.590999 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 896/1774] 10.222.0.66 () {60 vars in 1143 bytes} [Mon Jul 18 02:35:32 2022] POST /identity/v3/users => generated 309 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:35:32.590999 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b98754-5565-4814-aa4b-4d46992acfcf admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:32.595688 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-e8b98754-5565-4814-aa4b-4d46992acfcf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:32.631300 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-55a68abf-8acb-4bd2-9371-7651a4f9126c admin admin] The token cache is being invalidate because project a1adfe26f31649cab9688e5c766ed9da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 02:35:32.631300 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-55a68abf-8acb-4bd2-9371-7651a4f9126c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:32.631300 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-55a68abf-8acb-4bd2-9371-7651a4f9126c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:32.631300 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 879/1775] 10.222.0.66 () {58 vars in 1232 bytes} [Mon Jul 18 02:35:31 2022] DELETE /identity/v3/projects/a1adfe26f31649cab9688e5c766ed9da => generated 0 bytes in 714 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:35:32.632241 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a68abf-8acb-4bd2-9371-7651a4f9126c admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:32.633615 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-55a68abf-8acb-4bd2-9371-7651a4f9126c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:32.655669 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b98754-5565-4814-aa4b-4d46992acfcf admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:32.751021 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:35:32.842974 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a68abf-8acb-4bd2-9371-7651a4f9126c admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:32.866489 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d77a078-c7d6-440a-98b3-776541754ad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:32.867214 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d77a078-c7d6-440a-98b3-776541754ad8 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:32.867530 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d77a078-c7d6-440a-98b3-776541754ad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:32.867783 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d77a078-c7d6-440a-98b3-776541754ad8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:32.868518 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d77a078-c7d6-440a-98b3-776541754ad8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:32.873551 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d77a078-c7d6-440a-98b3-776541754ad8 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:32.885070 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf448cb0-1a2b-4241-97d8-28e00ac4e726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:32.887744 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf448cb0-1a2b-4241-97d8-28e00ac4e726 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:32.888237 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf448cb0-1a2b-4241-97d8-28e00ac4e726 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:32.888669 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf448cb0-1a2b-4241-97d8-28e00ac4e726 admin admin] PATH_INFO: `/v3/users/d9c2326a653a49d89f03331a9888ff60` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:32.891792 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf448cb0-1a2b-4241-97d8-28e00ac4e726 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9c2326a653a49d89f03331a9888ff60)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:32.898015 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf448cb0-1a2b-4241-97d8-28e00ac4e726 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:32.901228 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 880/1776] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:35:32 2022] GET /identity/v3/roles => generated 1420 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:35:32.903925 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c301ea-424e-4ed6-bf1d-25ec5d735840 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:32.904582 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c301ea-424e-4ed6-bf1d-25ec5d735840 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:32.906301 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c301ea-424e-4ed6-bf1d-25ec5d735840 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:32.964217 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-19c301ea-424e-4ed6-bf1d-25ec5d735840 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:35:32.965767 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-19c301ea-424e-4ed6-bf1d-25ec5d735840 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:33.009145 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 881/1777] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:35:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:35:33.010696 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c301ea-424e-4ed6-bf1d-25ec5d735840 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:33.042985 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c301ea-424e-4ed6-bf1d-25ec5d735840 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:33.054428 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8caa598-e88c-49d4-bce5-7f94f0392dd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:33.065006 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8caa598-e88c-49d4-bce5-7f94f0392dd7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:33.065006 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8caa598-e88c-49d4-bce5-7f94f0392dd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:33.065006 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8caa598-e88c-49d4-bce5-7f94f0392dd7 admin admin] PATH_INFO: `/v3/projects/d3c89fd7e68c4f6bacceac80724de4d7/users/fdbc6d910d32469898697f372da64cc7/roles/d4aa3324f0c84887b41d0dc4acf7f30e` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:33.076261 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8caa598-e88c-49d4-bce5-7f94f0392dd7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3c89fd7e68c4f6bacceac80724de4d7, user_id=fdbc6d910d32469898697f372da64cc7, role_id=d4aa3324f0c84887b41d0dc4acf7f30e)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:33.079568 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8caa598-e88c-49d4-bce5-7f94f0392dd7 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:33.545980 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 882/1778] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:35:33 2022] PUT /identity/v3/projects/d3c89fd7e68c4f6bacceac80724de4d7/users/fdbc6d910d32469898697f372da64cc7/roles/d4aa3324f0c84887b41d0dc4acf7f30e => generated 0 bytes in 536 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:35:33.557096 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7c4146-029b-4f83-903f-f00a9e34a6fe None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:33.557096 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7c4146-029b-4f83-903f-f00a9e34a6fe None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:33.557096 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7c4146-029b-4f83-903f-f00a9e34a6fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:33.573089 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-0b7c4146-029b-4f83-903f-f00a9e34a6fe None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:35:33.575607 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-0b7c4146-029b-4f83-903f-f00a9e34a6fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:33.697184 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 883/1779] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:35:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:35:33.699209 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7c4146-029b-4f83-903f-f00a9e34a6fe None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:33.709832 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-cf448cb0-1a2b-4241-97d8-28e00ac4e726 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9c2326a653a49d89f03331a9888ff60'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:33.721110 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7c4146-029b-4f83-903f-f00a9e34a6fe None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:33.724201 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62710756-c2ba-4cea-b472-c76cec28b6d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:33.727552 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62710756-c2ba-4cea-b472-c76cec28b6d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:33.728205 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62710756-c2ba-4cea-b472-c76cec28b6d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:33.728879 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62710756-c2ba-4cea-b472-c76cec28b6d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:33.729985 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62710756-c2ba-4cea-b472-c76cec28b6d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:33.732617 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62710756-c2ba-4cea-b472-c76cec28b6d5 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:33.747480 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 884/1780] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:35:33 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:35:33.748887 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62710756-c2ba-4cea-b472-c76cec28b6d5 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:33.773969 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62710756-c2ba-4cea-b472-c76cec28b6d5 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:33.785667 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-cf448cb0-1a2b-4241-97d8-28e00ac4e726 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9c2326a653a49d89f03331a9888ff60'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:33.787025 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559d8fa6-40f0-4a88-b4c2-2b514a5a51df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:33.788183 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559d8fa6-40f0-4a88-b4c2-2b514a5a51df admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:33.788483 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559d8fa6-40f0-4a88-b4c2-2b514a5a51df admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:33.788760 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559d8fa6-40f0-4a88-b4c2-2b514a5a51df admin admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:33.789538 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559d8fa6-40f0-4a88-b4c2-2b514a5a51df admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:35:33.789845 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559d8fa6-40f0-4a88-b4c2-2b514a5a51df admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:33.791764 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559d8fa6-40f0-4a88-b4c2-2b514a5a51df admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:33.799142 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-cf448cb0-1a2b-4241-97d8-28e00ac4e726 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9c2326a653a49d89f03331a9888ff60'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:33.801108 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-cf448cb0-1a2b-4241-97d8-28e00ac4e726 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9c2326a653a49d89f03331a9888ff60'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:33.806825 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 885/1781] 10.222.0.66 () {58 vars in 1152 bytes} [Mon Jul 18 02:35:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:35:33.807930 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559d8fa6-40f0-4a88-b4c2-2b514a5a51df admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:33.810580 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-559d8fa6-40f0-4a88-b4c2-2b514a5a51df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:33.839931 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-cf448cb0-1a2b-4241-97d8-28e00ac4e726 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9c2326a653a49d89f03331a9888ff60'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:33.984738 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559d8fa6-40f0-4a88-b4c2-2b514a5a51df admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:33.986825 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 897/1782] 10.222.0.66 () {58 vars in 1223 bytes} [Mon Jul 18 02:35:32 2022] DELETE /identity/v3/users/d9c2326a653a49d89f03331a9888ff60 => generated 0 bytes in 1261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:35:33.986825 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf448cb0-1a2b-4241-97d8-28e00ac4e726 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:33.986825 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-cf448cb0-1a2b-4241-97d8-28e00ac4e726 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:33.987445 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4dffc0-93a0-45fd-8507-351c8d425a22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:33.987445 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4dffc0-93a0-45fd-8507-351c8d425a22 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:33.987445 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4dffc0-93a0-45fd-8507-351c8d425a22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:33.987445 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4dffc0-93a0-45fd-8507-351c8d425a22 admin admin] PATH_INFO: `/v3/projects/d3c89fd7e68c4f6bacceac80724de4d7/users/fdbc6d910d32469898697f372da64cc7/roles/d4aa3324f0c84887b41d0dc4acf7f30e` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:33.987779 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4dffc0-93a0-45fd-8507-351c8d425a22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3c89fd7e68c4f6bacceac80724de4d7, user_id=fdbc6d910d32469898697f372da64cc7, role_id=d4aa3324f0c84887b41d0dc4acf7f30e)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:33.987779 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4dffc0-93a0-45fd-8507-351c8d425a22 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:33.987779 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 886/1783] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:35:33 2022] PUT /identity/v3/projects/d3c89fd7e68c4f6bacceac80724de4d7/users/fdbc6d910d32469898697f372da64cc7/roles/d4aa3324f0c84887b41d0dc4acf7f30e => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:35:33.988024 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf448cb0-1a2b-4241-97d8-28e00ac4e726 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:33.999617 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c01ea5-5ff7-44c6-a582-05454c340d2d None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:34.000058 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c01ea5-5ff7-44c6-a582-05454c340d2d None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:34.000420 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c01ea5-5ff7-44c6-a582-05454c340d2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:34.058390 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ac3407-e571-4833-85ab-a19186b84c2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:34.059131 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ac3407-e571-4833-85ab-a19186b84c2b admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:34.061963 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ac3407-e571-4833-85ab-a19186b84c2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:34.062306 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ac3407-e571-4833-85ab-a19186b84c2b admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:34.065076 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ac3407-e571-4833-85ab-a19186b84c2b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:34.071885 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ac3407-e571-4833-85ab-a19186b84c2b admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:34.086669 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-d1c01ea5-5ff7-44c6-a582-05454c340d2d None None] MFA Rules not processed for user `fdbc6d910d32469898697f372da64cc7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:35:34.088677 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-d1c01ea5-5ff7-44c6-a582-05454c340d2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:34.236765 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 887/1784] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:35:33 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:35:34.253157 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:35:34.263350 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c01ea5-5ff7-44c6-a582-05454c340d2d None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:34.265596 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 898/1785] 10.222.0.66 () {60 vars in 1152 bytes} [Mon Jul 18 02:35:33 2022] POST /identity/v3/projects => generated 361 bytes in 417 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:35:34.267948 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-d1c01ea5-5ff7-44c6-a582-05454c340d2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:34.270126 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ac3407-e571-4833-85ab-a19186b84c2b admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:34.416815 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ac3407-e571-4833-85ab-a19186b84c2b admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:34.424528 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c01ea5-5ff7-44c6-a582-05454c340d2d None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:34.429386 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e996c9-f59e-4389-811a-11690944c243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:34.429594 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e996c9-f59e-4389-811a-11690944c243 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:34.429962 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e996c9-f59e-4389-811a-11690944c243 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:34.430161 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e996c9-f59e-4389-811a-11690944c243 admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:34.430938 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e996c9-f59e-4389-811a-11690944c243 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:34.436596 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e996c9-f59e-4389-811a-11690944c243 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:34.536634 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acded56-0d4e-4562-a27c-d705cbad8b3b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd49388facdf446885e55b45ef91f01e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:34.539735 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acded56-0d4e-4562-a27c-d705cbad8b3b service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:34.539735 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acded56-0d4e-4562-a27c-d705cbad8b3b service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:34.539735 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acded56-0d4e-4562-a27c-d705cbad8b3b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:34.620083 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 899/1786] 10.222.0.66 () {60 vars in 1143 bytes} [Mon Jul 18 02:35:34 2022] POST /identity/v3/users => generated 281 bytes in 351 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:35:34.625105 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e996c9-f59e-4389-811a-11690944c243 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:34.636083 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e996c9-f59e-4389-811a-11690944c243 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:34.647873 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d48d12-1671-4e01-b0c3-5d9427330789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:34.651602 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d48d12-1671-4e01-b0c3-5d9427330789 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:34.653791 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d48d12-1671-4e01-b0c3-5d9427330789 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:34.654267 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d48d12-1671-4e01-b0c3-5d9427330789 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:34.655398 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d48d12-1671-4e01-b0c3-5d9427330789 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:34.666690 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d48d12-1671-4e01-b0c3-5d9427330789 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:34.843684 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 900/1787] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:35:34 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:35:34.845173 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acded56-0d4e-4562-a27c-d705cbad8b3b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:34.845173 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acded56-0d4e-4562-a27c-d705cbad8b3b service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:34.845173 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 888/1788] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:35:34 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 484 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 02:35:34.852966 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d48d12-1671-4e01-b0c3-5d9427330789 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:34.898583 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d48d12-1671-4e01-b0c3-5d9427330789 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:34.929706 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac88cc2b-fb76-41a1-be75-cf63c61865e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:34.942630 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac88cc2b-fb76-41a1-be75-cf63c61865e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:34.942630 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac88cc2b-fb76-41a1-be75-cf63c61865e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:34.942630 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac88cc2b-fb76-41a1-be75-cf63c61865e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:34.942630 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac88cc2b-fb76-41a1-be75-cf63c61865e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:34.942630 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac88cc2b-fb76-41a1-be75-cf63c61865e5 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:35.029215 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acded56-0d4e-4562-a27c-d705cbad8b3b service cinder] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:35.029708 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 901/1789] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:35:34 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:35:35.040230 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acded56-0d4e-4562-a27c-d705cbad8b3b service cinder] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:35.062930 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bf3ec5-0527-4fd6-aad8-c267f7ae8ab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:35.063874 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bf3ec5-0527-4fd6-aad8-c267f7ae8ab5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:35.064143 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bf3ec5-0527-4fd6-aad8-c267f7ae8ab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:35.064407 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bf3ec5-0527-4fd6-aad8-c267f7ae8ab5 admin admin] PATH_INFO: `/v3/projects/cebda7fa1256482983c6eb0ae6d3799a/users/618d295143664e4bab6fc07784f18585/roles/65feabf3b27944a8b01a86c8ac608457` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:35.085726 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bf3ec5-0527-4fd6-aad8-c267f7ae8ab5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cebda7fa1256482983c6eb0ae6d3799a, user_id=618d295143664e4bab6fc07784f18585, role_id=65feabf3b27944a8b01a86c8ac608457)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:35.090635 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bf3ec5-0527-4fd6-aad8-c267f7ae8ab5 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:35.132195 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac88cc2b-fb76-41a1-be75-cf63c61865e5 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:35.145867 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac88cc2b-fb76-41a1-be75-cf63c61865e5 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:35.177951 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1919b3f0-48a1-4959-ad4b-eceab1ae887a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:35.179316 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1919b3f0-48a1-4959-ad4b-eceab1ae887a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:35.179761 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1919b3f0-48a1-4959-ad4b-eceab1ae887a admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:35.180245 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1919b3f0-48a1-4959-ad4b-eceab1ae887a admin admin] PATH_INFO: `/v3/users/234b4142711d4304ba355c4ed0192aa5` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:35.181941 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1919b3f0-48a1-4959-ad4b-eceab1ae887a admin admin] RBAC: Authorizing `identity:delete_user(user_id=234b4142711d4304ba355c4ed0192aa5)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:35.185510 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1919b3f0-48a1-4959-ad4b-eceab1ae887a admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:35.235508 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 889/1790] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:35:35 2022] PUT /identity/v3/projects/cebda7fa1256482983c6eb0ae6d3799a/users/618d295143664e4bab6fc07784f18585/roles/65feabf3b27944a8b01a86c8ac608457 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:35:35.252065 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c1f421-50ef-4c47-83dd-8bd9d2048b48 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:35.252521 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c1f421-50ef-4c47-83dd-8bd9d2048b48 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:35.253602 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c1f421-50ef-4c47-83dd-8bd9d2048b48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:35.331927 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-d3c1f421-50ef-4c47-83dd-8bd9d2048b48 None None] MFA Rules not processed for user `618d295143664e4bab6fc07784f18585`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:35:35.336721 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-d3c1f421-50ef-4c47-83dd-8bd9d2048b48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:35.457685 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-1919b3f0-48a1-4959-ad4b-eceab1ae887a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '234b4142711d4304ba355c4ed0192aa5'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:35.533759 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:35:35.548454 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 890/1791] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:35:35 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:35:35.553921 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c1f421-50ef-4c47-83dd-8bd9d2048b48 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:35.567038 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-1919b3f0-48a1-4959-ad4b-eceab1ae887a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '234b4142711d4304ba355c4ed0192aa5'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:35.586427 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-1919b3f0-48a1-4959-ad4b-eceab1ae887a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '234b4142711d4304ba355c4ed0192aa5'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:35.586427 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-1919b3f0-48a1-4959-ad4b-eceab1ae887a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '234b4142711d4304ba355c4ed0192aa5'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:35.655502 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-1919b3f0-48a1-4959-ad4b-eceab1ae887a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '234b4142711d4304ba355c4ed0192aa5'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:35.682258 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 902/1792] 10.222.0.66 () {58 vars in 1223 bytes} [Mon Jul 18 02:35:35 2022] DELETE /identity/v3/users/234b4142711d4304ba355c4ed0192aa5 => generated 0 bytes in 548 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:35:35.687910 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1919b3f0-48a1-4959-ad4b-eceab1ae887a admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:35.691792 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-1919b3f0-48a1-4959-ad4b-eceab1ae887a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:35.795748 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c1f421-50ef-4c47-83dd-8bd9d2048b48 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:35.919302 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1919b3f0-48a1-4959-ad4b-eceab1ae887a admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:35.930651 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817781b1-57ee-4c6d-9cb1-08deb9906a50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:35.931796 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817781b1-57ee-4c6d-9cb1-08deb9906a50 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:35.932133 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817781b1-57ee-4c6d-9cb1-08deb9906a50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:35.932437 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817781b1-57ee-4c6d-9cb1-08deb9906a50 admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:35.933816 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817781b1-57ee-4c6d-9cb1-08deb9906a50 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:35.937818 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817781b1-57ee-4c6d-9cb1-08deb9906a50 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:35.937938 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699741e9-8f88-49c6-8ce2-e04f639c408f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:35.937938 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699741e9-8f88-49c6-8ce2-e04f639c408f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:35.937938 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699741e9-8f88-49c6-8ce2-e04f639c408f admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:35.940673 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699741e9-8f88-49c6-8ce2-e04f639c408f admin admin] PATH_INFO: `/v3/projects/1d493928bffa467d8ca39fe0f5403cbf` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:35.946717 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699741e9-8f88-49c6-8ce2-e04f639c408f admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d493928bffa467d8ca39fe0f5403cbf)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:35.948856 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699741e9-8f88-49c6-8ce2-e04f639c408f admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:36.131461 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 891/1793] 10.222.0.66 () {60 vars in 1152 bytes} [Mon Jul 18 02:35:35 2022] POST /identity/v3/projects => generated 363 bytes in 579 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 02:35:36.138303 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2b1984-762f-4470-bf07-37ff4b825a78 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:36.138303 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2b1984-762f-4470-bf07-37ff4b825a78 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:36.138303 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2b1984-762f-4470-bf07-37ff4b825a78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:36.162535 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-2f2b1984-762f-4470-bf07-37ff4b825a78 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:35:36.164115 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-2f2b1984-762f-4470-bf07-37ff4b825a78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:36.244333 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 892/1794] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:35:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:35:36.245369 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2b1984-762f-4470-bf07-37ff4b825a78 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:36.247685 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-2f2b1984-762f-4470-bf07-37ff4b825a78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:36.281825 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2b1984-762f-4470-bf07-37ff4b825a78 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:36.289183 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279bf3ac-04ae-40c5-a4c4-9dbd5a12098a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:36.290156 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279bf3ac-04ae-40c5-a4c4-9dbd5a12098a admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:36.290750 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279bf3ac-04ae-40c5-a4c4-9dbd5a12098a admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:36.291529 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279bf3ac-04ae-40c5-a4c4-9dbd5a12098a admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:36.292888 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279bf3ac-04ae-40c5-a4c4-9dbd5a12098a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:36.295255 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279bf3ac-04ae-40c5-a4c4-9dbd5a12098a admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:36.465872 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-699741e9-8f88-49c6-8ce2-e04f639c408f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d493928bffa467d8ca39fe0f5403cbf', 'request_id': 'req-699741e9-8f88-49c6-8ce2-e04f639c408f'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:36.465872 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-699741e9-8f88-49c6-8ce2-e04f639c408f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d493928bffa467d8ca39fe0f5403cbf', 'request_id': 'req-699741e9-8f88-49c6-8ce2-e04f639c408f'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:36.986981 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 893/1795] 10.222.0.66 () {60 vars in 1143 bytes} [Mon Jul 18 02:35:36 2022] POST /identity/v3/users => generated 288 bytes in 742 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:35:36.988221 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279bf3ac-04ae-40c5-a4c4-9dbd5a12098a admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:36.992944 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-279bf3ac-04ae-40c5-a4c4-9dbd5a12098a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:37.109880 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-699741e9-8f88-49c6-8ce2-e04f639c408f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d493928bffa467d8ca39fe0f5403cbf', 'request_id': 'req-699741e9-8f88-49c6-8ce2-e04f639c408f'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:37.135625 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-699741e9-8f88-49c6-8ce2-e04f639c408f admin admin] The token cache is being invalidate because project 1d493928bffa467d8ca39fe0f5403cbf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 02:35:37.135897 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-699741e9-8f88-49c6-8ce2-e04f639c408f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:37.144764 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279bf3ac-04ae-40c5-a4c4-9dbd5a12098a admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:37.154829 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-699741e9-8f88-49c6-8ce2-e04f639c408f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:37.160280 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 903/1796] 10.222.0.66 () {58 vars in 1232 bytes} [Mon Jul 18 02:35:35 2022] DELETE /identity/v3/projects/1d493928bffa467d8ca39fe0f5403cbf => generated 0 bytes in 1477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:35:37.182309 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699741e9-8f88-49c6-8ce2-e04f639c408f admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:37.182309 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-699741e9-8f88-49c6-8ce2-e04f639c408f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:37.252556 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f68d272-9528-4859-a28e-bd1b66dbc6b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eaf19a259c2741cea375a38763e33cdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:37.253646 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f68d272-9528-4859-a28e-bd1b66dbc6b1 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:37.253942 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f68d272-9528-4859-a28e-bd1b66dbc6b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:37.254247 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f68d272-9528-4859-a28e-bd1b66dbc6b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:37.287116 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-1f68d272-9528-4859-a28e-bd1b66dbc6b1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:37.358978 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699741e9-8f88-49c6-8ce2-e04f639c408f admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:37.366559 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae1605c-a03e-4bb1-a47a-05c9e2045612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:37.368349 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae1605c-a03e-4bb1-a47a-05c9e2045612 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:37.368995 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae1605c-a03e-4bb1-a47a-05c9e2045612 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:37.369653 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae1605c-a03e-4bb1-a47a-05c9e2045612 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:37.371576 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae1605c-a03e-4bb1-a47a-05c9e2045612 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:37.374045 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae1605c-a03e-4bb1-a47a-05c9e2045612 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:37.396154 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 904/1797] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:35:37 2022] GET /identity/v3/roles => generated 1420 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:35:37.400186 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b401e5-66da-43bd-80ed-f35c23adc04d None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:37.400705 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b401e5-66da-43bd-80ed-f35c23adc04d None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:37.402265 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b401e5-66da-43bd-80ed-f35c23adc04d None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:37.406291 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 905/1798] 10.222.0.66 () {54 vars in 921 bytes} [Mon Jul 18 02:35:37 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:35:37.411550 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844d0ac9-9ffe-45a1-a8a1-6ed875a36617 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:37.430759 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844d0ac9-9ffe-45a1-a8a1-6ed875a36617 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:37.431454 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844d0ac9-9ffe-45a1-a8a1-6ed875a36617 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:37.485238 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-844d0ac9-9ffe-45a1-a8a1-6ed875a36617 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:35:37.486907 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-844d0ac9-9ffe-45a1-a8a1-6ed875a36617 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:37.492374 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f68d272-9528-4859-a28e-bd1b66dbc6b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:37.495167 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f68d272-9528-4859-a28e-bd1b66dbc6b1 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:37.508090 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:35:37.511012 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 894/1799] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:35:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 523 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:35:37.511153 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 906/1800] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:35:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:35:37.518533 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f68d272-9528-4859-a28e-bd1b66dbc6b1 service neutron] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:37.520746 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5533392-5ccd-4e15-a82a-9e7ed06a584b None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:37.521047 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5533392-5ccd-4e15-a82a-9e7ed06a584b None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:37.521391 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5533392-5ccd-4e15-a82a-9e7ed06a584b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:37.528105 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f68d272-9528-4859-a28e-bd1b66dbc6b1 service neutron] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:37.534508 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac933df-2278-42d8-96d6-f16b24624224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:37.535679 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac933df-2278-42d8-96d6-f16b24624224 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:37.536221 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac933df-2278-42d8-96d6-f16b24624224 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:37.537078 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac933df-2278-42d8-96d6-f16b24624224 admin admin] PATH_INFO: `/v3/projects/39e44736542148688d31ca5535ed0c57/users/163cde81e59a4773896561e47a9237b6/roles/d4aa3324f0c84887b41d0dc4acf7f30e` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:37.571777 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-d5533392-5ccd-4e15-a82a-9e7ed06a584b None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:35:37.573689 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-d5533392-5ccd-4e15-a82a-9e7ed06a584b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:37.582653 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac933df-2278-42d8-96d6-f16b24624224 admin admin] RBAC: Authorizing `identity:create_grant(project_id=39e44736542148688d31ca5535ed0c57, user_id=163cde81e59a4773896561e47a9237b6, role_id=d4aa3324f0c84887b41d0dc4acf7f30e)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:37.587735 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac933df-2278-42d8-96d6-f16b24624224 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:37.611201 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 907/1801] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:35:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:35:37.615093 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f118be6-467c-4a1f-b9db-5e115cde1b44 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:37.615093 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f118be6-467c-4a1f-b9db-5e115cde1b44 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:37.615093 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f118be6-467c-4a1f-b9db-5e115cde1b44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:37.642413 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-0f118be6-467c-4a1f-b9db-5e115cde1b44 None None] MFA Rules not processed for user `80b138831bc54c7b986d50092d62d4af`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:35:37.644220 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-0f118be6-467c-4a1f-b9db-5e115cde1b44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:37.688863 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 895/1802] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:35:37 2022] PUT /identity/v3/projects/39e44736542148688d31ca5535ed0c57/users/163cde81e59a4773896561e47a9237b6/roles/d4aa3324f0c84887b41d0dc4acf7f30e => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:35:37.688863 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac933df-2278-42d8-96d6-f16b24624224 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:37.765632 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 908/1803] 10.222.0.66 () {58 vars in 1018 bytes} [Mon Jul 18 02:35:37 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:35:37.766609 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f118be6-467c-4a1f-b9db-5e115cde1b44 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:37.786909 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f118be6-467c-4a1f-b9db-5e115cde1b44 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:37.827796 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac933df-2278-42d8-96d6-f16b24624224 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:37.840351 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19582995-3ce4-4150-ab92-b584ae26cef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:37.841437 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19582995-3ce4-4150-ab92-b584ae26cef1 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:37.841967 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19582995-3ce4-4150-ab92-b584ae26cef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:37.842720 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19582995-3ce4-4150-ab92-b584ae26cef1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:37.843762 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19582995-3ce4-4150-ab92-b584ae26cef1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:35:37.845907 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19582995-3ce4-4150-ab92-b584ae26cef1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:37.853395 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19582995-3ce4-4150-ab92-b584ae26cef1 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:37.868977 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 896/1804] 10.222.0.66 () {58 vars in 1152 bytes} [Mon Jul 18 02:35:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:35:37.875350 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19582995-3ce4-4150-ab92-b584ae26cef1 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:37.882386 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaefece-db28-44a5-8535-e518d6092634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:37.883159 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaefece-db28-44a5-8535-e518d6092634 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:37.883725 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaefece-db28-44a5-8535-e518d6092634 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:37.884166 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaefece-db28-44a5-8535-e518d6092634 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:37.885040 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aaefece-db28-44a5-8535-e518d6092634 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:37.889203 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aaefece-db28-44a5-8535-e518d6092634 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:37.889524 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19582995-3ce4-4150-ab92-b584ae26cef1 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:37.897622 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6392432-a60e-4e04-a5cf-a1f301050f2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:37.898745 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6392432-a60e-4e04-a5cf-a1f301050f2c admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:37.899099 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6392432-a60e-4e04-a5cf-a1f301050f2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:37.899475 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6392432-a60e-4e04-a5cf-a1f301050f2c admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:37.900966 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6392432-a60e-4e04-a5cf-a1f301050f2c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:37.904193 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6392432-a60e-4e04-a5cf-a1f301050f2c admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:37.906411 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 909/1805] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:35:37 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:35:37.910155 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaefece-db28-44a5-8535-e518d6092634 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:37.926245 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaefece-db28-44a5-8535-e518d6092634 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:37.934044 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c68ccd-beb9-41db-b57b-91ea8ea83a25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:37.935315 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c68ccd-beb9-41db-b57b-91ea8ea83a25 admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:37.935775 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c68ccd-beb9-41db-b57b-91ea8ea83a25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:37.936197 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c68ccd-beb9-41db-b57b-91ea8ea83a25 admin admin] PATH_INFO: `/v3/projects/39e44736542148688d31ca5535ed0c57/users/163cde81e59a4773896561e47a9237b6/roles/d4aa3324f0c84887b41d0dc4acf7f30e` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:37.939698 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c68ccd-beb9-41db-b57b-91ea8ea83a25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=39e44736542148688d31ca5535ed0c57, user_id=163cde81e59a4773896561e47a9237b6, role_id=d4aa3324f0c84887b41d0dc4acf7f30e)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:37.943985 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c68ccd-beb9-41db-b57b-91ea8ea83a25 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:37.974800 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 910/1806] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:35:37 2022] PUT /identity/v3/projects/39e44736542148688d31ca5535ed0c57/users/163cde81e59a4773896561e47a9237b6/roles/d4aa3324f0c84887b41d0dc4acf7f30e => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:35:37.978524 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c68ccd-beb9-41db-b57b-91ea8ea83a25 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:37.978524 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-f8c68ccd-beb9-41db-b57b-91ea8ea83a25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:38.044029 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 897/1807] 10.222.0.66 () {60 vars in 1152 bytes} [Mon Jul 18 02:35:37 2022] POST /identity/v3/projects => generated 407 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:35:38.045539 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec1bebb-b5ae-412b-a603-49f287dced12 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:38.047054 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec1bebb-b5ae-412b-a603-49f287dced12 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:38.048559 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec1bebb-b5ae-412b-a603-49f287dced12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:38.088466 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-3ec1bebb-b5ae-412b-a603-49f287dced12 None None] MFA Rules not processed for user `163cde81e59a4773896561e47a9237b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:35:38.098199 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-3ec1bebb-b5ae-412b-a603-49f287dced12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:38.123792 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c68ccd-beb9-41db-b57b-91ea8ea83a25 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:38.198558 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 898/1808] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:35:38 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:35:38.201298 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec1bebb-b5ae-412b-a603-49f287dced12 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:38.239038 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41374da-5e33-4002-b632-2746b3a0d52a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80b138831bc54c7b986d50092d62d4af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:38.239623 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41374da-5e33-4002-b632-2746b3a0d52a service nova] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:38.239911 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41374da-5e33-4002-b632-2746b3a0d52a service nova] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:38.240378 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41374da-5e33-4002-b632-2746b3a0d52a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:38.252728 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41374da-5e33-4002-b632-2746b3a0d52a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:38.255548 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41374da-5e33-4002-b632-2746b3a0d52a service nova] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:38.268898 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 911/1809] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:35:37 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:35:38.269926 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41374da-5e33-4002-b632-2746b3a0d52a service nova] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:38.271814 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-f41374da-5e33-4002-b632-2746b3a0d52a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:38.306101 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec1bebb-b5ae-412b-a603-49f287dced12 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:38.334957 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fd74e0-d5b6-4fce-8335-dd6bb9f6c3e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:38.336856 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fd74e0-d5b6-4fce-8335-dd6bb9f6c3e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:38.339316 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fd74e0-d5b6-4fce-8335-dd6bb9f6c3e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:38.339891 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fd74e0-d5b6-4fce-8335-dd6bb9f6c3e6 admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:38.341114 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fd74e0-d5b6-4fce-8335-dd6bb9f6c3e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:38.346164 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fd74e0-d5b6-4fce-8335-dd6bb9f6c3e6 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:38.352927 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:35:38.399394 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41374da-5e33-4002-b632-2746b3a0d52a service nova] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:38.403347 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86419afe-6468-4ef8-a52f-9fd00227fb91 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd49388facdf446885e55b45ef91f01e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:38.404292 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86419afe-6468-4ef8-a52f-9fd00227fb91 service cinder] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:38.404784 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86419afe-6468-4ef8-a52f-9fd00227fb91 service cinder] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:38.405355 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86419afe-6468-4ef8-a52f-9fd00227fb91 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:38.417199 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86419afe-6468-4ef8-a52f-9fd00227fb91 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:38.420421 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86419afe-6468-4ef8-a52f-9fd00227fb91 service cinder] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:38.434751 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 912/1810] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:35:38 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:35:38.435845 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86419afe-6468-4ef8-a52f-9fd00227fb91 service cinder] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:38.437327 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-86419afe-6468-4ef8-a52f-9fd00227fb91 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:38.456537 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 899/1811] 10.222.0.66 () {60 vars in 1143 bytes} [Mon Jul 18 02:35:38 2022] POST /identity/v3/users => generated 304 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:35:38.456767 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86419afe-6468-4ef8-a52f-9fd00227fb91 service cinder] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:38.456880 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fd74e0-d5b6-4fce-8335-dd6bb9f6c3e6 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:38.459117 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-f7fd74e0-d5b6-4fce-8335-dd6bb9f6c3e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:38.461897 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e489063f-a155-4dfc-b77e-50237829c462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:38.463006 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e489063f-a155-4dfc-b77e-50237829c462 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:38.463393 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e489063f-a155-4dfc-b77e-50237829c462 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:38.463775 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e489063f-a155-4dfc-b77e-50237829c462 admin admin] PATH_INFO: `/v3/projects/4a7cc87017b94cc68c7ae0304dadadf4` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:38.466395 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e489063f-a155-4dfc-b77e-50237829c462 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a7cc87017b94cc68c7ae0304dadadf4)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:38.468496 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e489063f-a155-4dfc-b77e-50237829c462 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:38.503458 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fd74e0-d5b6-4fce-8335-dd6bb9f6c3e6 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:38.509447 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99eb5cb-6a01-4b0e-935a-68078fc9c879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:38.510521 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99eb5cb-6a01-4b0e-935a-68078fc9c879 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:38.510814 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99eb5cb-6a01-4b0e-935a-68078fc9c879 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:38.511094 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99eb5cb-6a01-4b0e-935a-68078fc9c879 admin admin] PATH_INFO: `/v3/users/abd6b032a11047a88019d785e60b715a` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:38.512873 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99eb5cb-6a01-4b0e-935a-68078fc9c879 admin admin] RBAC: Authorizing `identity:delete_user(user_id=abd6b032a11047a88019d785e60b715a)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:38.525409 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99eb5cb-6a01-4b0e-935a-68078fc9c879 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:38.810745 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-d99eb5cb-6a01-4b0e-935a-68078fc9c879 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'abd6b032a11047a88019d785e60b715a'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:38.819154 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-e489063f-a155-4dfc-b77e-50237829c462 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a7cc87017b94cc68c7ae0304dadadf4', 'request_id': 'req-e489063f-a155-4dfc-b77e-50237829c462'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:38.827345 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-e489063f-a155-4dfc-b77e-50237829c462 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a7cc87017b94cc68c7ae0304dadadf4', 'request_id': 'req-e489063f-a155-4dfc-b77e-50237829c462'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:38.937884 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-d99eb5cb-6a01-4b0e-935a-68078fc9c879 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'abd6b032a11047a88019d785e60b715a'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:38.943553 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-d99eb5cb-6a01-4b0e-935a-68078fc9c879 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'abd6b032a11047a88019d785e60b715a'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:38.953296 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-d99eb5cb-6a01-4b0e-935a-68078fc9c879 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'abd6b032a11047a88019d785e60b715a'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:38.954039 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-d99eb5cb-6a01-4b0e-935a-68078fc9c879 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'abd6b032a11047a88019d785e60b715a'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:38.979264 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 900/1812] 10.222.0.66 () {58 vars in 1223 bytes} [Mon Jul 18 02:35:38 2022] DELETE /identity/v3/users/abd6b032a11047a88019d785e60b715a => generated 0 bytes in 525 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:35:38.980131 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99eb5cb-6a01-4b0e-935a-68078fc9c879 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:38.981958 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-d99eb5cb-6a01-4b0e-935a-68078fc9c879 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:39.035019 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-e489063f-a155-4dfc-b77e-50237829c462 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a7cc87017b94cc68c7ae0304dadadf4', 'request_id': 'req-e489063f-a155-4dfc-b77e-50237829c462'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:39.050550 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-e489063f-a155-4dfc-b77e-50237829c462 admin admin] The token cache is being invalidate because project 4a7cc87017b94cc68c7ae0304dadadf4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 02:35:39.050900 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-e489063f-a155-4dfc-b77e-50237829c462 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:39.051583 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-e489063f-a155-4dfc-b77e-50237829c462 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:39.057114 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 913/1813] 10.222.0.66 () {58 vars in 1232 bytes} [Mon Jul 18 02:35:38 2022] DELETE /identity/v3/projects/4a7cc87017b94cc68c7ae0304dadadf4 => generated 0 bytes in 618 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:35:39.057414 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99eb5cb-6a01-4b0e-935a-68078fc9c879 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:39.059268 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5872c73-216e-426b-a880-652973058d3b None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:39.059554 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5872c73-216e-426b-a880-652973058d3b None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:39.059854 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5872c73-216e-426b-a880-652973058d3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:39.079188 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8d3cd6-0c85-4c25-8170-5240637987c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:39.081196 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8d3cd6-0c85-4c25-8170-5240637987c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:39.083596 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8d3cd6-0c85-4c25-8170-5240637987c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:39.084000 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8d3cd6-0c85-4c25-8170-5240637987c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:39.085001 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8d3cd6-0c85-4c25-8170-5240637987c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:39.087715 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8d3cd6-0c85-4c25-8170-5240637987c7 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:39.088845 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-c5872c73-216e-426b-a880-652973058d3b None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:35:39.094380 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-c5872c73-216e-426b-a880-652973058d3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:39.101336 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 901/1814] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:35:38 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:35:39.103261 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8d3cd6-0c85-4c25-8170-5240637987c7 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:39.105686 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-fd8d3cd6-0c85-4c25-8170-5240637987c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:39.110205 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 914/1815] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:35:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:35:39.117763 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5872c73-216e-426b-a880-652973058d3b None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:39.117763 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-c5872c73-216e-426b-a880-652973058d3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:39.124492 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8d3cd6-0c85-4c25-8170-5240637987c7 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:39.126508 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5872c73-216e-426b-a880-652973058d3b None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:39.132752 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86421bd3-5fcb-49ce-81ae-fa13b5cb5a3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:39.133286 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7143d1-8dd5-4d10-b709-b291ad34acb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:39.133442 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86421bd3-5fcb-49ce-81ae-fa13b5cb5a3c admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:39.133732 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86421bd3-5fcb-49ce-81ae-fa13b5cb5a3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:39.134009 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86421bd3-5fcb-49ce-81ae-fa13b5cb5a3c admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:39.134773 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86421bd3-5fcb-49ce-81ae-fa13b5cb5a3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:39.135425 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7143d1-8dd5-4d10-b709-b291ad34acb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:39.135908 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7143d1-8dd5-4d10-b709-b291ad34acb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:39.136848 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7143d1-8dd5-4d10-b709-b291ad34acb5 admin admin] PATH_INFO: `/v3/users/fdbc6d910d32469898697f372da64cc7` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:39.137144 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86421bd3-5fcb-49ce-81ae-fa13b5cb5a3c admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:39.140030 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7143d1-8dd5-4d10-b709-b291ad34acb5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fdbc6d910d32469898697f372da64cc7)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:39.142290 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7143d1-8dd5-4d10-b709-b291ad34acb5 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:39.148319 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 915/1816] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:35:39 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:35:39.149330 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86421bd3-5fcb-49ce-81ae-fa13b5cb5a3c admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:39.153495 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-86421bd3-5fcb-49ce-81ae-fa13b5cb5a3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:39.239517 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86421bd3-5fcb-49ce-81ae-fa13b5cb5a3c admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:39.247350 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c99fc7c-a715-428a-9361-8b956f690ce4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eaf19a259c2741cea375a38763e33cdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:39.247350 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c99fc7c-a715-428a-9361-8b956f690ce4 service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:39.247350 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c99fc7c-a715-428a-9361-8b956f690ce4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:39.247350 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c99fc7c-a715-428a-9361-8b956f690ce4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:39.254373 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c99fc7c-a715-428a-9361-8b956f690ce4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:39.255823 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c99fc7c-a715-428a-9361-8b956f690ce4 service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:39.266805 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 916/1817] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:35:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:35:39.267522 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c99fc7c-a715-428a-9361-8b956f690ce4 service neutron] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:39.286591 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c99fc7c-a715-428a-9361-8b956f690ce4 service neutron] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:39.295516 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184fa8a5-6e98-4da2-8a77-bd90ecb3c0e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:39.296453 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184fa8a5-6e98-4da2-8a77-bd90ecb3c0e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:39.296737 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184fa8a5-6e98-4da2-8a77-bd90ecb3c0e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:39.297016 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184fa8a5-6e98-4da2-8a77-bd90ecb3c0e2 admin admin] PATH_INFO: `/v3/projects/955171086a7844a2a6abc66fb56692f7/users/1c1a3587e5a844228d482ffe977cf573/roles/65feabf3b27944a8b01a86c8ac608457` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:39.316624 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184fa8a5-6e98-4da2-8a77-bd90ecb3c0e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=955171086a7844a2a6abc66fb56692f7, user_id=1c1a3587e5a844228d482ffe977cf573, role_id=65feabf3b27944a8b01a86c8ac608457)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:39.319395 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184fa8a5-6e98-4da2-8a77-bd90ecb3c0e2 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:39.342110 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-9c7143d1-8dd5-4d10-b709-b291ad34acb5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fdbc6d910d32469898697f372da64cc7'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:39.430689 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 917/1818] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:35:39 2022] PUT /identity/v3/projects/955171086a7844a2a6abc66fb56692f7/users/1c1a3587e5a844228d482ffe977cf573/roles/65feabf3b27944a8b01a86c8ac608457 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:35:39.441690 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc08911-6432-48f4-8bd7-217cd5a93ca5 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:39.441990 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc08911-6432-48f4-8bd7-217cd5a93ca5 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:39.442300 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc08911-6432-48f4-8bd7-217cd5a93ca5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:39.455604 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-9c7143d1-8dd5-4d10-b709-b291ad34acb5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fdbc6d910d32469898697f372da64cc7'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:39.463988 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-9c7143d1-8dd5-4d10-b709-b291ad34acb5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fdbc6d910d32469898697f372da64cc7'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:39.475541 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-cdc08911-6432-48f4-8bd7-217cd5a93ca5 None None] MFA Rules not processed for user `1c1a3587e5a844228d482ffe977cf573`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:35:39.477016 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-cdc08911-6432-48f4-8bd7-217cd5a93ca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:39.482582 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-9c7143d1-8dd5-4d10-b709-b291ad34acb5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fdbc6d910d32469898697f372da64cc7'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:39.483291 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-9c7143d1-8dd5-4d10-b709-b291ad34acb5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fdbc6d910d32469898697f372da64cc7'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:39.500715 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 902/1819] 10.222.0.66 () {58 vars in 1223 bytes} [Mon Jul 18 02:35:39 2022] DELETE /identity/v3/users/fdbc6d910d32469898697f372da64cc7 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:35:39.549091 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 918/1820] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:35:39 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 02:35:39.553487 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7143d1-8dd5-4d10-b709-b291ad34acb5 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:39.555073 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-9c7143d1-8dd5-4d10-b709-b291ad34acb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:39.587683 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc08911-6432-48f4-8bd7-217cd5a93ca5 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:39.592423 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-cdc08911-6432-48f4-8bd7-217cd5a93ca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:39.676734 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7143d1-8dd5-4d10-b709-b291ad34acb5 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:39.679731 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc08911-6432-48f4-8bd7-217cd5a93ca5 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:39.681790 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248d4534-033a-4e0e-90d3-e960c9e37523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:39.682722 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248d4534-033a-4e0e-90d3-e960c9e37523 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:39.683005 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248d4534-033a-4e0e-90d3-e960c9e37523 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:39.683847 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248d4534-033a-4e0e-90d3-e960c9e37523 admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:39.684974 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248d4534-033a-4e0e-90d3-e960c9e37523 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:39.686237 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ce4997-9d61-416b-9ead-fd45f7765cba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:39.686960 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248d4534-033a-4e0e-90d3-e960c9e37523 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:39.687469 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ce4997-9d61-416b-9ead-fd45f7765cba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:39.687855 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ce4997-9d61-416b-9ead-fd45f7765cba admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:39.688414 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ce4997-9d61-416b-9ead-fd45f7765cba admin admin] PATH_INFO: `/v3/users/618d295143664e4bab6fc07784f18585` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:39.689916 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ce4997-9d61-416b-9ead-fd45f7765cba admin admin] RBAC: Authorizing `identity:delete_user(user_id=618d295143664e4bab6fc07784f18585)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:39.692261 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ce4997-9d61-416b-9ead-fd45f7765cba admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:39.862220 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 903/1821] 10.222.0.66 () {60 vars in 1152 bytes} [Mon Jul 18 02:35:39 2022] POST /identity/v3/projects => generated 407 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:35:39.866696 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248d4534-033a-4e0e-90d3-e960c9e37523 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:39.965124 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248d4534-033a-4e0e-90d3-e960c9e37523 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:39.970643 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fd34ec-6c5a-4646-a4a6-c26c5507caa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:39.971342 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fd34ec-6c5a-4646-a4a6-c26c5507caa2 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:39.971694 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fd34ec-6c5a-4646-a4a6-c26c5507caa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:39.972133 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fd34ec-6c5a-4646-a4a6-c26c5507caa2 admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:39.972844 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fd34ec-6c5a-4646-a4a6-c26c5507caa2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:39.975039 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fd34ec-6c5a-4646-a4a6-c26c5507caa2 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:40.126374 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 904/1822] 10.222.0.66 () {60 vars in 1143 bytes} [Mon Jul 18 02:35:39 2022] POST /identity/v3/users => generated 310 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 02:35:40.136846 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-59ce4997-9d61-416b-9ead-fd45f7765cba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '618d295143664e4bab6fc07784f18585'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:40.142627 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fd34ec-6c5a-4646-a4a6-c26c5507caa2 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:40.166475 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fd34ec-6c5a-4646-a4a6-c26c5507caa2 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:40.195919 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c04db8-2b68-43ae-8c92-5ad87131d524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:40.198577 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c04db8-2b68-43ae-8c92-5ad87131d524 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:40.198798 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c04db8-2b68-43ae-8c92-5ad87131d524 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:40.199083 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c04db8-2b68-43ae-8c92-5ad87131d524 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:40.201278 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c04db8-2b68-43ae-8c92-5ad87131d524 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:40.206970 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c04db8-2b68-43ae-8c92-5ad87131d524 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:40.211673 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-59ce4997-9d61-416b-9ead-fd45f7765cba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '618d295143664e4bab6fc07784f18585'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:40.221339 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 905/1823] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:35:40 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:35:40.222005 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-59ce4997-9d61-416b-9ead-fd45f7765cba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '618d295143664e4bab6fc07784f18585'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:40.222627 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c04db8-2b68-43ae-8c92-5ad87131d524 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:40.224970 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-59ce4997-9d61-416b-9ead-fd45f7765cba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '618d295143664e4bab6fc07784f18585'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:40.241054 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-88c04db8-2b68-43ae-8c92-5ad87131d524 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:40.255868 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-59ce4997-9d61-416b-9ead-fd45f7765cba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '618d295143664e4bab6fc07784f18585'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:40.270580 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 919/1824] 10.222.0.66 () {58 vars in 1223 bytes} [Mon Jul 18 02:35:39 2022] DELETE /identity/v3/users/618d295143664e4bab6fc07784f18585 => generated 0 bytes in 684 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:35:40.271723 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ce4997-9d61-416b-9ead-fd45f7765cba admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:40.275025 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-59ce4997-9d61-416b-9ead-fd45f7765cba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:40.277539 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c04db8-2b68-43ae-8c92-5ad87131d524 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:40.296463 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:35:40.378125 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ce4997-9d61-416b-9ead-fd45f7765cba admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:40.386525 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13256790-7bfe-49e1-a19b-970f99173eec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:40.387604 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13256790-7bfe-49e1-a19b-970f99173eec admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:40.388524 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13256790-7bfe-49e1-a19b-970f99173eec admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:40.389491 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13256790-7bfe-49e1-a19b-970f99173eec admin admin] PATH_INFO: `/v3/projects/cb397225f2a3464f917242ab02841844/users/b94783458b4f4156a75400800b2327de/roles/d4aa3324f0c84887b41d0dc4acf7f30e` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:40.395544 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c36de4-c74c-4e59-9543-0f1ff3a410df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:40.397057 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c36de4-c74c-4e59-9543-0f1ff3a410df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:40.397560 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c36de4-c74c-4e59-9543-0f1ff3a410df admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:40.398048 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c36de4-c74c-4e59-9543-0f1ff3a410df admin admin] PATH_INFO: `/v3/projects/d3c89fd7e68c4f6bacceac80724de4d7` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:40.399728 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c36de4-c74c-4e59-9543-0f1ff3a410df admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3c89fd7e68c4f6bacceac80724de4d7)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:40.402928 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c36de4-c74c-4e59-9543-0f1ff3a410df admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:40.407778 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13256790-7bfe-49e1-a19b-970f99173eec admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb397225f2a3464f917242ab02841844, user_id=b94783458b4f4156a75400800b2327de, role_id=d4aa3324f0c84887b41d0dc4acf7f30e)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:40.411990 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13256790-7bfe-49e1-a19b-970f99173eec admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:40.504736 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 920/1825] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:35:40 2022] PUT /identity/v3/projects/cb397225f2a3464f917242ab02841844/users/b94783458b4f4156a75400800b2327de/roles/d4aa3324f0c84887b41d0dc4acf7f30e => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:35:40.506321 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c366a45d-cec9-4db6-bb07-2df58d22e155 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:40.509207 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c366a45d-cec9-4db6-bb07-2df58d22e155 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:40.509207 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c366a45d-cec9-4db6-bb07-2df58d22e155 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:40.528709 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-c366a45d-cec9-4db6-bb07-2df58d22e155 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:35:40.529927 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-c366a45d-cec9-4db6-bb07-2df58d22e155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:40.584484 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-69c36de4-c74c-4e59-9543-0f1ff3a410df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3c89fd7e68c4f6bacceac80724de4d7', 'request_id': 'req-69c36de4-c74c-4e59-9543-0f1ff3a410df'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:40.591340 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-69c36de4-c74c-4e59-9543-0f1ff3a410df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3c89fd7e68c4f6bacceac80724de4d7', 'request_id': 'req-69c36de4-c74c-4e59-9543-0f1ff3a410df'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:40.598644 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-69c36de4-c74c-4e59-9543-0f1ff3a410df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3c89fd7e68c4f6bacceac80724de4d7', 'request_id': 'req-69c36de4-c74c-4e59-9543-0f1ff3a410df'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:40.606216 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 921/1826] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:35:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:35:40.612282 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c366a45d-cec9-4db6-bb07-2df58d22e155 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:40.616722 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-c366a45d-cec9-4db6-bb07-2df58d22e155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:40.630570 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c366a45d-cec9-4db6-bb07-2df58d22e155 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:40.678174 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-69c36de4-c74c-4e59-9543-0f1ff3a410df admin admin] The token cache is being invalidate because project d3c89fd7e68c4f6bacceac80724de4d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 02:35:40.678594 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-69c36de4-c74c-4e59-9543-0f1ff3a410df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:40.679320 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-69c36de4-c74c-4e59-9543-0f1ff3a410df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:40.685019 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 906/1827] 10.222.0.66 () {58 vars in 1232 bytes} [Mon Jul 18 02:35:40 2022] DELETE /identity/v3/projects/d3c89fd7e68c4f6bacceac80724de4d7 => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:35:40.685019 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c36de4-c74c-4e59-9543-0f1ff3a410df admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:40.685019 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-69c36de4-c74c-4e59-9543-0f1ff3a410df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:40.705674 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2153044-a6ae-4994-870b-59266bfaf456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:40.706771 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2153044-a6ae-4994-870b-59266bfaf456 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:40.707128 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2153044-a6ae-4994-870b-59266bfaf456 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:40.707474 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2153044-a6ae-4994-870b-59266bfaf456 admin admin] PATH_INFO: `/v3/projects/9179e44bf9d74d50ae488bdeca0723df` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:40.708753 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2153044-a6ae-4994-870b-59266bfaf456 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9179e44bf9d74d50ae488bdeca0723df)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:40.710819 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2153044-a6ae-4994-870b-59266bfaf456 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:40.756662 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c36de4-c74c-4e59-9543-0f1ff3a410df admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:40.761380 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cac208-8ff5-4a2f-bfd3-b5f615f27039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:40.762048 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cac208-8ff5-4a2f-bfd3-b5f615f27039 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:40.762360 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cac208-8ff5-4a2f-bfd3-b5f615f27039 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:40.762680 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cac208-8ff5-4a2f-bfd3-b5f615f27039 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:40.763438 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cac208-8ff5-4a2f-bfd3-b5f615f27039 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:40.765053 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cac208-8ff5-4a2f-bfd3-b5f615f27039 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:40.775406 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 907/1828] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:35:40 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:35:40.780065 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cac208-8ff5-4a2f-bfd3-b5f615f27039 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:40.780065 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-f8cac208-8ff5-4a2f-bfd3-b5f615f27039 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:40.832810 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cac208-8ff5-4a2f-bfd3-b5f615f27039 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:40.836437 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97b6f82-af37-4ed8-996b-052e5481d567 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eaf19a259c2741cea375a38763e33cdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:40.837081 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97b6f82-af37-4ed8-996b-052e5481d567 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:40.837367 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97b6f82-af37-4ed8-996b-052e5481d567 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:40.837646 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97b6f82-af37-4ed8-996b-052e5481d567 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:40.839950 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-a97b6f82-af37-4ed8-996b-052e5481d567 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:40.852191 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97b6f82-af37-4ed8-996b-052e5481d567 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:40.852533 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-b2153044-a6ae-4994-870b-59266bfaf456 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9179e44bf9d74d50ae488bdeca0723df', 'request_id': 'req-b2153044-a6ae-4994-870b-59266bfaf456'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:40.853562 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97b6f82-af37-4ed8-996b-052e5481d567 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:40.858906 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-b2153044-a6ae-4994-870b-59266bfaf456 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9179e44bf9d74d50ae488bdeca0723df', 'request_id': 'req-b2153044-a6ae-4994-870b-59266bfaf456'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:40.864372 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 908/1829] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:35:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:35:40.865638 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f84efdf-e687-4026-9d3e-93a916d18c8c None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:40.865963 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f84efdf-e687-4026-9d3e-93a916d18c8c None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:40.866242 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f84efdf-e687-4026-9d3e-93a916d18c8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:40.884326 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-6f84efdf-e687-4026-9d3e-93a916d18c8c None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:35:40.885572 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-6f84efdf-e687-4026-9d3e-93a916d18c8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:40.925888 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-b2153044-a6ae-4994-870b-59266bfaf456 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9179e44bf9d74d50ae488bdeca0723df', 'request_id': 'req-b2153044-a6ae-4994-870b-59266bfaf456'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:40.939450 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-b2153044-a6ae-4994-870b-59266bfaf456 admin admin] The token cache is being invalidate because project 9179e44bf9d74d50ae488bdeca0723df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 02:35:40.939735 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-b2153044-a6ae-4994-870b-59266bfaf456 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:40.940416 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-b2153044-a6ae-4994-870b-59266bfaf456 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:40.945315 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 922/1830] 10.222.0.66 () {58 vars in 1232 bytes} [Mon Jul 18 02:35:40 2022] DELETE /identity/v3/projects/9179e44bf9d74d50ae488bdeca0723df => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:35:40.946184 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2153044-a6ae-4994-870b-59266bfaf456 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:40.947583 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-b2153044-a6ae-4994-870b-59266bfaf456 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:40.950582 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 909/1831] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:35:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:35:40.951633 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f84efdf-e687-4026-9d3e-93a916d18c8c None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:40.957947 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-6f84efdf-e687-4026-9d3e-93a916d18c8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:40.967195 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2153044-a6ae-4994-870b-59266bfaf456 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:40.971983 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f84efdf-e687-4026-9d3e-93a916d18c8c None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:40.972122 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff38e33-db1f-4fb2-8aed-368002e3e6a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:40.972878 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff38e33-db1f-4fb2-8aed-368002e3e6a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:40.973162 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff38e33-db1f-4fb2-8aed-368002e3e6a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:40.973440 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff38e33-db1f-4fb2-8aed-368002e3e6a8 admin admin] PATH_INFO: `/v3/projects/cb397225f2a3464f917242ab02841844/users/b94783458b4f4156a75400800b2327de/roles/d4aa3324f0c84887b41d0dc4acf7f30e` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:40.975408 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff38e33-db1f-4fb2-8aed-368002e3e6a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb397225f2a3464f917242ab02841844, user_id=b94783458b4f4156a75400800b2327de, role_id=d4aa3324f0c84887b41d0dc4acf7f30e)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:40.977307 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105fc2ca-c79c-44f3-9eb3-bb8c9b58a5a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:40.978283 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105fc2ca-c79c-44f3-9eb3-bb8c9b58a5a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:40.978410 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff38e33-db1f-4fb2-8aed-368002e3e6a8 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:40.978702 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105fc2ca-c79c-44f3-9eb3-bb8c9b58a5a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:40.979042 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105fc2ca-c79c-44f3-9eb3-bb8c9b58a5a7 admin admin] PATH_INFO: `/v3/users/41c43858514f4d82bf826a58177b5eb8` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:40.980926 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105fc2ca-c79c-44f3-9eb3-bb8c9b58a5a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=41c43858514f4d82bf826a58177b5eb8)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:40.983229 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105fc2ca-c79c-44f3-9eb3-bb8c9b58a5a7 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:40.990606 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 923/1832] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:35:40 2022] PUT /identity/v3/projects/cb397225f2a3464f917242ab02841844/users/b94783458b4f4156a75400800b2327de/roles/d4aa3324f0c84887b41d0dc4acf7f30e => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:35:40.994210 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f35e2fa-e139-4b97-9b5b-c75465fcaa96 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:40.994210 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f35e2fa-e139-4b97-9b5b-c75465fcaa96 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:40.994210 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f35e2fa-e139-4b97-9b5b-c75465fcaa96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:41.012007 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-8f35e2fa-e139-4b97-9b5b-c75465fcaa96 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:35:41.013587 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-8f35e2fa-e139-4b97-9b5b-c75465fcaa96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:41.091196 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 924/1833] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:35:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:35:41.092592 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad9f75b-b8a4-4e91-94eb-aedb2f5d8b2b None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:41.092888 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad9f75b-b8a4-4e91-94eb-aedb2f5d8b2b None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:41.093168 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad9f75b-b8a4-4e91-94eb-aedb2f5d8b2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:41.121711 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-fad9f75b-b8a4-4e91-94eb-aedb2f5d8b2b None None] MFA Rules not processed for user `b94783458b4f4156a75400800b2327de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:35:41.122997 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-fad9f75b-b8a4-4e91-94eb-aedb2f5d8b2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:41.209782 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-105fc2ca-c79c-44f3-9eb3-bb8c9b58a5a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41c43858514f4d82bf826a58177b5eb8'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:41.215398 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 925/1834] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:35:41 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:35:41.217556 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad9f75b-b8a4-4e91-94eb-aedb2f5d8b2b None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:41.235070 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad9f75b-b8a4-4e91-94eb-aedb2f5d8b2b None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:41.247560 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52a7182-c33a-4b6d-b7cf-426448d25222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:41.248590 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52a7182-c33a-4b6d-b7cf-426448d25222 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:41.249014 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52a7182-c33a-4b6d-b7cf-426448d25222 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:41.249391 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52a7182-c33a-4b6d-b7cf-426448d25222 admin admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:41.250415 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52a7182-c33a-4b6d-b7cf-426448d25222 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:35:41.251017 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52a7182-c33a-4b6d-b7cf-426448d25222 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:41.253003 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52a7182-c33a-4b6d-b7cf-426448d25222 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:41.274483 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 926/1835] 10.222.0.66 () {58 vars in 1152 bytes} [Mon Jul 18 02:35:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:35:41.274483 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52a7182-c33a-4b6d-b7cf-426448d25222 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:41.274483 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-e52a7182-c33a-4b6d-b7cf-426448d25222 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:41.297905 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-105fc2ca-c79c-44f3-9eb3-bb8c9b58a5a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41c43858514f4d82bf826a58177b5eb8'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:41.298613 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-105fc2ca-c79c-44f3-9eb3-bb8c9b58a5a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41c43858514f4d82bf826a58177b5eb8'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:41.319787 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-105fc2ca-c79c-44f3-9eb3-bb8c9b58a5a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41c43858514f4d82bf826a58177b5eb8'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:41.320743 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-105fc2ca-c79c-44f3-9eb3-bb8c9b58a5a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41c43858514f4d82bf826a58177b5eb8'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:41.363315 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 910/1836] 10.222.0.66 () {58 vars in 1223 bytes} [Mon Jul 18 02:35:40 2022] DELETE /identity/v3/users/41c43858514f4d82bf826a58177b5eb8 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:35:41.364504 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105fc2ca-c79c-44f3-9eb3-bb8c9b58a5a7 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:41.372523 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52a7182-c33a-4b6d-b7cf-426448d25222 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:41.376780 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-105fc2ca-c79c-44f3-9eb3-bb8c9b58a5a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:41.500615 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d196c3-5634-4194-9b0e-f02fe396a290 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd49388facdf446885e55b45ef91f01e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:41.501415 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d196c3-5634-4194-9b0e-f02fe396a290 service cinder] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:41.502888 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105fc2ca-c79c-44f3-9eb3-bb8c9b58a5a7 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:41.503332 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d196c3-5634-4194-9b0e-f02fe396a290 service cinder] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:41.503418 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d196c3-5634-4194-9b0e-f02fe396a290 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:41.505475 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-64d196c3-5634-4194-9b0e-f02fe396a290 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:41.508486 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c9ede8-260a-4ad8-98b9-a97a3242dcb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:41.509269 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c9ede8-260a-4ad8-98b9-a97a3242dcb1 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:41.509787 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c9ede8-260a-4ad8-98b9-a97a3242dcb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:41.510199 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c9ede8-260a-4ad8-98b9-a97a3242dcb1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:41.511779 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c9ede8-260a-4ad8-98b9-a97a3242dcb1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:41.513895 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c9ede8-260a-4ad8-98b9-a97a3242dcb1 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:41.588758 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d196c3-5634-4194-9b0e-f02fe396a290 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:41.590629 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d196c3-5634-4194-9b0e-f02fe396a290 service cinder] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:41.601802 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 927/1837] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:35:41 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:35:41.603097 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d196c3-5634-4194-9b0e-f02fe396a290 service cinder] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:41.604705 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-64d196c3-5634-4194-9b0e-f02fe396a290 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:41.619662 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d196c3-5634-4194-9b0e-f02fe396a290 service cinder] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:41.626021 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe31d44-55a0-48ea-a229-554dc4c2fe23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:41.629976 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe31d44-55a0-48ea-a229-554dc4c2fe23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:41.629976 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe31d44-55a0-48ea-a229-554dc4c2fe23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:41.629976 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe31d44-55a0-48ea-a229-554dc4c2fe23 admin admin] PATH_INFO: `/v3/projects/cebda7fa1256482983c6eb0ae6d3799a` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:41.629976 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe31d44-55a0-48ea-a229-554dc4c2fe23 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cebda7fa1256482983c6eb0ae6d3799a)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:41.630897 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 911/1838] 10.222.0.66 () {60 vars in 1152 bytes} [Mon Jul 18 02:35:41 2022] POST /identity/v3/projects => generated 385 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:35:41.631612 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe31d44-55a0-48ea-a229-554dc4c2fe23 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:41.632901 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c9ede8-260a-4ad8-98b9-a97a3242dcb1 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:41.709626 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:35:41.721037 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c9ede8-260a-4ad8-98b9-a97a3242dcb1 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:41.728764 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128d9bf7-3a64-446a-8978-a16e37b7dd88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:41.729454 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128d9bf7-3a64-446a-8978-a16e37b7dd88 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:41.730284 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128d9bf7-3a64-446a-8978-a16e37b7dd88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:41.730284 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128d9bf7-3a64-446a-8978-a16e37b7dd88 admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:41.730987 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128d9bf7-3a64-446a-8978-a16e37b7dd88 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:41.733045 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128d9bf7-3a64-446a-8978-a16e37b7dd88 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:41.809937 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-9fe31d44-55a0-48ea-a229-554dc4c2fe23 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cebda7fa1256482983c6eb0ae6d3799a', 'request_id': 'req-9fe31d44-55a0-48ea-a229-554dc4c2fe23'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:41.817235 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-9fe31d44-55a0-48ea-a229-554dc4c2fe23 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cebda7fa1256482983c6eb0ae6d3799a', 'request_id': 'req-9fe31d44-55a0-48ea-a229-554dc4c2fe23'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:41.848189 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 912/1839] 10.222.0.66 () {60 vars in 1143 bytes} [Mon Jul 18 02:35:41 2022] POST /identity/v3/users => generated 293 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:35:41.852771 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128d9bf7-3a64-446a-8978-a16e37b7dd88 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:41.864117 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128d9bf7-3a64-446a-8978-a16e37b7dd88 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:41.926934 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-9fe31d44-55a0-48ea-a229-554dc4c2fe23 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cebda7fa1256482983c6eb0ae6d3799a', 'request_id': 'req-9fe31d44-55a0-48ea-a229-554dc4c2fe23'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:41.945304 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44690a30-e6ff-4d62-8bfd-d29f951ad50f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:41.946336 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44690a30-e6ff-4d62-8bfd-d29f951ad50f admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:41.946537 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44690a30-e6ff-4d62-8bfd-d29f951ad50f admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:41.946750 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-9fe31d44-55a0-48ea-a229-554dc4c2fe23 admin admin] The token cache is being invalidate because project cebda7fa1256482983c6eb0ae6d3799a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 02:35:41.946951 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44690a30-e6ff-4d62-8bfd-d29f951ad50f admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:41.947343 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-9fe31d44-55a0-48ea-a229-554dc4c2fe23 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:41.948295 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-9fe31d44-55a0-48ea-a229-554dc4c2fe23 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:41.948421 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44690a30-e6ff-4d62-8bfd-d29f951ad50f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:41.950806 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44690a30-e6ff-4d62-8bfd-d29f951ad50f admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:41.951320 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 928/1840] 10.222.0.66 () {58 vars in 1232 bytes} [Mon Jul 18 02:35:41 2022] DELETE /identity/v3/projects/cebda7fa1256482983c6eb0ae6d3799a => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:35:41.955504 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe31d44-55a0-48ea-a229-554dc4c2fe23 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:41.957741 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-9fe31d44-55a0-48ea-a229-554dc4c2fe23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:41.975099 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 913/1841] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:35:41 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 02:35:41.977563 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe31d44-55a0-48ea-a229-554dc4c2fe23 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:41.979505 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44690a30-e6ff-4d62-8bfd-d29f951ad50f admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:41.982790 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-44690a30-e6ff-4d62-8bfd-d29f951ad50f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:41.985897 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005168be-945b-4e55-a71b-cc1e89bc8294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:41.987029 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005168be-945b-4e55-a71b-cc1e89bc8294 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:41.987913 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005168be-945b-4e55-a71b-cc1e89bc8294 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:41.987913 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005168be-945b-4e55-a71b-cc1e89bc8294 admin admin] PATH_INFO: `/v3/users/163cde81e59a4773896561e47a9237b6` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:41.989430 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005168be-945b-4e55-a71b-cc1e89bc8294 admin admin] RBAC: Authorizing `identity:delete_user(user_id=163cde81e59a4773896561e47a9237b6)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:41.991982 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005168be-945b-4e55-a71b-cc1e89bc8294 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:41.997932 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44690a30-e6ff-4d62-8bfd-d29f951ad50f admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:42.007726 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d496a1da-dce2-4336-857f-917c7832cbdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:42.008423 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d496a1da-dce2-4336-857f-917c7832cbdc admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:42.008743 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d496a1da-dce2-4336-857f-917c7832cbdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:42.009069 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d496a1da-dce2-4336-857f-917c7832cbdc admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:42.009867 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d496a1da-dce2-4336-857f-917c7832cbdc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:42.011707 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d496a1da-dce2-4336-857f-917c7832cbdc admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:42.028850 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 914/1842] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:35:41 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:35:42.033401 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d496a1da-dce2-4336-857f-917c7832cbdc admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:42.047829 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d496a1da-dce2-4336-857f-917c7832cbdc admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:42.053460 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110361ff-586b-4239-b572-340c52253dbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:42.055068 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110361ff-586b-4239-b572-340c52253dbb admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:42.055432 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110361ff-586b-4239-b572-340c52253dbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:42.055843 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110361ff-586b-4239-b572-340c52253dbb admin admin] PATH_INFO: `/v3/projects/887cb2ba061f404297533137b4c02a6e/users/9370658d43684988b40f263a01ceed34/roles/65feabf3b27944a8b01a86c8ac608457` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:42.076701 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110361ff-586b-4239-b572-340c52253dbb admin admin] RBAC: Authorizing `identity:create_grant(project_id=887cb2ba061f404297533137b4c02a6e, user_id=9370658d43684988b40f263a01ceed34, role_id=65feabf3b27944a8b01a86c8ac608457)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:42.079739 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110361ff-586b-4239-b572-340c52253dbb admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:42.176028 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 915/1843] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:35:42 2022] PUT /identity/v3/projects/887cb2ba061f404297533137b4c02a6e/users/9370658d43684988b40f263a01ceed34/roles/65feabf3b27944a8b01a86c8ac608457 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:35:42.177007 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110361ff-586b-4239-b572-340c52253dbb admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:42.179727 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-110361ff-586b-4239-b572-340c52253dbb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:42.218902 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-005168be-945b-4e55-a71b-cc1e89bc8294 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '163cde81e59a4773896561e47a9237b6'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:42.237262 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110361ff-586b-4239-b572-340c52253dbb admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:42.240897 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3407bfef-6e01-4a8a-9e07-6971c11a7b30 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd49388facdf446885e55b45ef91f01e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:42.241543 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3407bfef-6e01-4a8a-9e07-6971c11a7b30 service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:42.241829 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3407bfef-6e01-4a8a-9e07-6971c11a7b30 service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:42.242135 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3407bfef-6e01-4a8a-9e07-6971c11a7b30 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:42.243941 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-3407bfef-6e01-4a8a-9e07-6971c11a7b30 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:42.301280 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3407bfef-6e01-4a8a-9e07-6971c11a7b30 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:42.302854 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3407bfef-6e01-4a8a-9e07-6971c11a7b30 service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:42.303008 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-005168be-945b-4e55-a71b-cc1e89bc8294 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '163cde81e59a4773896561e47a9237b6'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:42.310539 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-005168be-945b-4e55-a71b-cc1e89bc8294 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '163cde81e59a4773896561e47a9237b6'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:42.311128 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-005168be-945b-4e55-a71b-cc1e89bc8294 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '163cde81e59a4773896561e47a9237b6'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:42.312293 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 916/1844] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:35:42 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:35:42.314792 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3407bfef-6e01-4a8a-9e07-6971c11a7b30 service cinder] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:42.314792 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-3407bfef-6e01-4a8a-9e07-6971c11a7b30 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:42.325515 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-005168be-945b-4e55-a71b-cc1e89bc8294 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '163cde81e59a4773896561e47a9237b6'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:42.331649 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 929/1845] 10.222.0.66 () {58 vars in 1223 bytes} [Mon Jul 18 02:35:41 2022] DELETE /identity/v3/users/163cde81e59a4773896561e47a9237b6 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:35:42.332768 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789c36bb-f148-4997-be74-d5c4db053b9f None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:42.333061 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789c36bb-f148-4997-be74-d5c4db053b9f None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:42.333343 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789c36bb-f148-4997-be74-d5c4db053b9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:42.385679 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-789c36bb-f148-4997-be74-d5c4db053b9f None None] MFA Rules not processed for user `9370658d43684988b40f263a01ceed34`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:35:42.387012 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-789c36bb-f148-4997-be74-d5c4db053b9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:42.404500 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3407bfef-6e01-4a8a-9e07-6971c11a7b30 service cinder] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:42.469426 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 930/1846] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:35:42 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:35:42.473543 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789c36bb-f148-4997-be74-d5c4db053b9f None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:42.474915 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3198186d-4a85-44c0-be5d-304ee0d4b446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:42.475784 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-789c36bb-f148-4997-be74-d5c4db053b9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:42.476632 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3198186d-4a85-44c0-be5d-304ee0d4b446 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:42.477307 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3198186d-4a85-44c0-be5d-304ee0d4b446 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:42.481069 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3198186d-4a85-44c0-be5d-304ee0d4b446 admin admin] PATH_INFO: `/v3/projects/4d08f76a482c48daa325fa385c2805c4` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:42.481069 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3198186d-4a85-44c0-be5d-304ee0d4b446 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d08f76a482c48daa325fa385c2805c4)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:42.482314 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3198186d-4a85-44c0-be5d-304ee0d4b446 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:42.496110 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789c36bb-f148-4997-be74-d5c4db053b9f None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:42.503613 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4db7e6-7792-430c-9488-7e00995abf56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:42.504730 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4db7e6-7792-430c-9488-7e00995abf56 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:42.505181 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4db7e6-7792-430c-9488-7e00995abf56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:42.505634 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4db7e6-7792-430c-9488-7e00995abf56 admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:42.506601 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4db7e6-7792-430c-9488-7e00995abf56 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:42.509637 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4db7e6-7792-430c-9488-7e00995abf56 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:42.626558 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 931/1847] 10.222.0.66 () {60 vars in 1152 bytes} [Mon Jul 18 02:35:42 2022] POST /identity/v3/projects => generated 387 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:35:42.632682 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4db7e6-7792-430c-9488-7e00995abf56 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:42.639775 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-3198186d-4a85-44c0-be5d-304ee0d4b446 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d08f76a482c48daa325fa385c2805c4', 'request_id': 'req-3198186d-4a85-44c0-be5d-304ee0d4b446'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:42.646204 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-3198186d-4a85-44c0-be5d-304ee0d4b446 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d08f76a482c48daa325fa385c2805c4', 'request_id': 'req-3198186d-4a85-44c0-be5d-304ee0d4b446'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:42.661585 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-3198186d-4a85-44c0-be5d-304ee0d4b446 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d08f76a482c48daa325fa385c2805c4', 'request_id': 'req-3198186d-4a85-44c0-be5d-304ee0d4b446'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:42.696092 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4db7e6-7792-430c-9488-7e00995abf56 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:42.701373 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a37c44-9c65-4e96-b0c1-072e2c05a4b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:42.702033 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a37c44-9c65-4e96-b0c1-072e2c05a4b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:42.702346 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a37c44-9c65-4e96-b0c1-072e2c05a4b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:42.702622 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a37c44-9c65-4e96-b0c1-072e2c05a4b3 admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:42.703303 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a37c44-9c65-4e96-b0c1-072e2c05a4b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:42.705220 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a37c44-9c65-4e96-b0c1-072e2c05a4b3 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:42.781630 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:35:42.860407 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-3198186d-4a85-44c0-be5d-304ee0d4b446 admin admin] The token cache is being invalidate because project 4d08f76a482c48daa325fa385c2805c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 02:35:42.867372 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-3198186d-4a85-44c0-be5d-304ee0d4b446 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:42.867372 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-3198186d-4a85-44c0-be5d-304ee0d4b446 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:42.867372 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 917/1848] 10.222.0.66 () {58 vars in 1232 bytes} [Mon Jul 18 02:35:42 2022] DELETE /identity/v3/projects/4d08f76a482c48daa325fa385c2805c4 => generated 0 bytes in 551 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:35:42.869090 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2a81b9-7ffa-4a30-94f7-d3c888646e0a None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:42.869488 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2a81b9-7ffa-4a30-94f7-d3c888646e0a None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:42.869873 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2a81b9-7ffa-4a30-94f7-d3c888646e0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:42.902385 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 932/1849] 10.222.0.66 () {60 vars in 1143 bytes} [Mon Jul 18 02:35:42 2022] POST /identity/v3/users => generated 294 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 02:35:42.905444 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-1a2a81b9-7ffa-4a30-94f7-d3c888646e0a None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:35:42.906787 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-1a2a81b9-7ffa-4a30-94f7-d3c888646e0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:42.910273 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a37c44-9c65-4e96-b0c1-072e2c05a4b3 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:42.912307 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-a4a37c44-9c65-4e96-b0c1-072e2c05a4b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:42.925355 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 918/1850] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:35:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:35:42.926959 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a29769-e2bf-497e-a066-e8d99eb4a1da None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:42.927274 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a29769-e2bf-497e-a066-e8d99eb4a1da None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:42.927528 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a29769-e2bf-497e-a066-e8d99eb4a1da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:42.930403 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a37c44-9c65-4e96-b0c1-072e2c05a4b3 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:42.936370 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220549b6-3165-4582-9be1-590909f346b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:42.937289 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220549b6-3165-4582-9be1-590909f346b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:42.938541 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220549b6-3165-4582-9be1-590909f346b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:42.938988 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220549b6-3165-4582-9be1-590909f346b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:42.939733 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220549b6-3165-4582-9be1-590909f346b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:42.952709 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-25a29769-e2bf-497e-a066-e8d99eb4a1da None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:35:42.953864 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220549b6-3165-4582-9be1-590909f346b1 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:42.953982 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-25a29769-e2bf-497e-a066-e8d99eb4a1da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:42.969942 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 933/1851] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:35:42 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 02:35:42.975790 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220549b6-3165-4582-9be1-590909f346b1 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:42.977550 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 919/1852] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:35:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:35:42.978529 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a29769-e2bf-497e-a066-e8d99eb4a1da None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:42.993572 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a29769-e2bf-497e-a066-e8d99eb4a1da None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:42.997201 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220549b6-3165-4582-9be1-590909f346b1 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:42.999502 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a79bf1-10b1-4045-a28b-eb1bc8c54ad7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:43.000170 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a79bf1-10b1-4045-a28b-eb1bc8c54ad7 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:43.000452 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a79bf1-10b1-4045-a28b-eb1bc8c54ad7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:43.000738 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a79bf1-10b1-4045-a28b-eb1bc8c54ad7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:43.001718 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a79bf1-10b1-4045-a28b-eb1bc8c54ad7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:35:43.001718 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a79bf1-10b1-4045-a28b-eb1bc8c54ad7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:43.003409 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a79bf1-10b1-4045-a28b-eb1bc8c54ad7 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:43.006558 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f569dfe0-aa4b-452b-a340-004bf9e78d7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:43.007211 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f569dfe0-aa4b-452b-a340-004bf9e78d7f admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:43.007498 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f569dfe0-aa4b-452b-a340-004bf9e78d7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:43.007919 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f569dfe0-aa4b-452b-a340-004bf9e78d7f admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:43.009116 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f569dfe0-aa4b-452b-a340-004bf9e78d7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:43.010678 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f569dfe0-aa4b-452b-a340-004bf9e78d7f admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:43.019640 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 920/1853] 10.222.0.66 () {58 vars in 1152 bytes} [Mon Jul 18 02:35:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:35:43.024421 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a79bf1-10b1-4045-a28b-eb1bc8c54ad7 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:43.027769 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 934/1854] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:35:42 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:35:43.032877 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f569dfe0-aa4b-452b-a340-004bf9e78d7f admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:43.039133 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a79bf1-10b1-4045-a28b-eb1bc8c54ad7 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:43.044955 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07510d93-882f-4b93-a8b9-8a1edca2fb6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:43.045622 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07510d93-882f-4b93-a8b9-8a1edca2fb6c admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:43.045839 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f569dfe0-aa4b-452b-a340-004bf9e78d7f admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:43.045953 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07510d93-882f-4b93-a8b9-8a1edca2fb6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:43.046198 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07510d93-882f-4b93-a8b9-8a1edca2fb6c admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:43.046897 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07510d93-882f-4b93-a8b9-8a1edca2fb6c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:43.048874 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07510d93-882f-4b93-a8b9-8a1edca2fb6c admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:43.052594 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd30a008-3bca-4bcb-bfbb-1d8a179b475a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:43.053517 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd30a008-3bca-4bcb-bfbb-1d8a179b475a admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:43.053806 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd30a008-3bca-4bcb-bfbb-1d8a179b475a admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:43.054112 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd30a008-3bca-4bcb-bfbb-1d8a179b475a admin admin] PATH_INFO: `/v3/projects/4521fed4fd5346faaa44e4201edbf601/users/3d80fb60b11c4a5a88136042ae549855/roles/65feabf3b27944a8b01a86c8ac608457` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:43.090827 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd30a008-3bca-4bcb-bfbb-1d8a179b475a admin admin] RBAC: Authorizing `identity:create_grant(project_id=4521fed4fd5346faaa44e4201edbf601, user_id=3d80fb60b11c4a5a88136042ae549855, role_id=65feabf3b27944a8b01a86c8ac608457)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:43.093859 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd30a008-3bca-4bcb-bfbb-1d8a179b475a admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:43.142592 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 921/1855] 10.222.0.66 () {60 vars in 1152 bytes} [Mon Jul 18 02:35:43 2022] POST /identity/v3/projects => generated 365 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 02:35:43.147578 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07510d93-882f-4b93-a8b9-8a1edca2fb6c admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:43.149378 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-07510d93-882f-4b93-a8b9-8a1edca2fb6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:43.223996 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 935/1856] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:35:43 2022] PUT /identity/v3/projects/4521fed4fd5346faaa44e4201edbf601/users/3d80fb60b11c4a5a88136042ae549855/roles/65feabf3b27944a8b01a86c8ac608457 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 02:35:43.227986 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd30a008-3bca-4bcb-bfbb-1d8a179b475a admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:43.235915 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07510d93-882f-4b93-a8b9-8a1edca2fb6c admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:43.365351 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068932a8-e2a3-49e6-8e68-b4249310a331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:43.366856 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068932a8-e2a3-49e6-8e68-b4249310a331 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:43.367146 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068932a8-e2a3-49e6-8e68-b4249310a331 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:43.367426 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068932a8-e2a3-49e6-8e68-b4249310a331 admin admin] PATH_INFO: `/v3/projects/39e44736542148688d31ca5535ed0c57` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:43.373181 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068932a8-e2a3-49e6-8e68-b4249310a331 admin admin] RBAC: Authorizing `identity:delete_project(project_id=39e44736542148688d31ca5535ed0c57)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:43.375202 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068932a8-e2a3-49e6-8e68-b4249310a331 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:43.377076 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd30a008-3bca-4bcb-bfbb-1d8a179b475a admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:43.390305 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a126eeb-5959-4780-889a-ea5a601e3357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:43.390996 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a126eeb-5959-4780-889a-ea5a601e3357 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:43.391277 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a126eeb-5959-4780-889a-ea5a601e3357 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:43.391555 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a126eeb-5959-4780-889a-ea5a601e3357 admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:43.392336 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a126eeb-5959-4780-889a-ea5a601e3357 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:43.395970 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a126eeb-5959-4780-889a-ea5a601e3357 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:43.507162 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 936/1857] 10.222.0.66 () {60 vars in 1143 bytes} [Mon Jul 18 02:35:43 2022] POST /identity/v3/users => generated 283 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:35:43.509405 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad8e5ec-5a24-4e6e-8083-d4abe6f20a4e None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:43.509809 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad8e5ec-5a24-4e6e-8083-d4abe6f20a4e None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:43.510952 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad8e5ec-5a24-4e6e-8083-d4abe6f20a4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:43.527715 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-068932a8-e2a3-49e6-8e68-b4249310a331 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39e44736542148688d31ca5535ed0c57', 'request_id': 'req-068932a8-e2a3-49e6-8e68-b4249310a331'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:43.545691 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-068932a8-e2a3-49e6-8e68-b4249310a331 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39e44736542148688d31ca5535ed0c57', 'request_id': 'req-068932a8-e2a3-49e6-8e68-b4249310a331'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:43.550440 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-fad8e5ec-5a24-4e6e-8083-d4abe6f20a4e None None] MFA Rules not processed for user `3d80fb60b11c4a5a88136042ae549855`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:35:43.551863 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-fad8e5ec-5a24-4e6e-8083-d4abe6f20a4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:43.556453 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-068932a8-e2a3-49e6-8e68-b4249310a331 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39e44736542148688d31ca5535ed0c57', 'request_id': 'req-068932a8-e2a3-49e6-8e68-b4249310a331'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:43.711469 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 937/1858] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:35:43 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:35:43.714901 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad8e5ec-5a24-4e6e-8083-d4abe6f20a4e None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:43.739173 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-068932a8-e2a3-49e6-8e68-b4249310a331 admin admin] The token cache is being invalidate because project 39e44736542148688d31ca5535ed0c57 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 02:35:43.739706 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-068932a8-e2a3-49e6-8e68-b4249310a331 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:43.742830 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-068932a8-e2a3-49e6-8e68-b4249310a331 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:43.748786 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 922/1859] 10.222.0.66 () {58 vars in 1232 bytes} [Mon Jul 18 02:35:43 2022] DELETE /identity/v3/projects/39e44736542148688d31ca5535ed0c57 => generated 0 bytes in 597 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:35:43.748786 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068932a8-e2a3-49e6-8e68-b4249310a331 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:43.750560 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-068932a8-e2a3-49e6-8e68-b4249310a331 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:43.840515 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad8e5ec-5a24-4e6e-8083-d4abe6f20a4e None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:43.848919 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250233e1-a679-4eac-9b0c-a8ab990de92e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:43.850667 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250233e1-a679-4eac-9b0c-a8ab990de92e admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:43.851264 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250233e1-a679-4eac-9b0c-a8ab990de92e admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:43.852370 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250233e1-a679-4eac-9b0c-a8ab990de92e admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:43.854324 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250233e1-a679-4eac-9b0c-a8ab990de92e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:43.856658 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250233e1-a679-4eac-9b0c-a8ab990de92e admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:43.890433 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 938/1860] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:35:43 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:35:43.896920 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068932a8-e2a3-49e6-8e68-b4249310a331 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:43.897060 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383beeda-8b98-4cb1-9f7b-e5e9aa8ee94b None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:43.897060 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383beeda-8b98-4cb1-9f7b-e5e9aa8ee94b None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:43.897060 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383beeda-8b98-4cb1-9f7b-e5e9aa8ee94b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:43.899268 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057b0e56-b553-4cc6-a0ab-e04c4c3903b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:43.900209 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057b0e56-b553-4cc6-a0ab-e04c4c3903b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:43.900681 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057b0e56-b553-4cc6-a0ab-e04c4c3903b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:43.901096 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057b0e56-b553-4cc6-a0ab-e04c4c3903b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:43.902016 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057b0e56-b553-4cc6-a0ab-e04c4c3903b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:43.904219 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057b0e56-b553-4cc6-a0ab-e04c4c3903b4 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:43.927582 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-383beeda-8b98-4cb1-9f7b-e5e9aa8ee94b None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:35:43.929410 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-383beeda-8b98-4cb1-9f7b-e5e9aa8ee94b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:43.947213 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 939/1861] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:35:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:35:43.949276 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae70f7b-978b-461d-a6fe-b4dffe3ec25d None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:43.949669 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae70f7b-978b-461d-a6fe-b4dffe3ec25d None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:43.950064 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae70f7b-978b-461d-a6fe-b4dffe3ec25d None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:43.951796 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 940/1862] 10.222.0.66 () {54 vars in 921 bytes} [Mon Jul 18 02:35:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:35:43.952619 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae70f7b-978b-461d-a6fe-b4dffe3ec25d None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:43.954714 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-0ae70f7b-978b-461d-a6fe-b4dffe3ec25d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:43.971860 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae70f7b-978b-461d-a6fe-b4dffe3ec25d None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:43.976529 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06791fee-dd0c-4711-92e4-122fe234bda7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:43.977306 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06791fee-dd0c-4711-92e4-122fe234bda7 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:43.977660 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06791fee-dd0c-4711-92e4-122fe234bda7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:43.978008 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06791fee-dd0c-4711-92e4-122fe234bda7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:43.978831 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06791fee-dd0c-4711-92e4-122fe234bda7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:43.981259 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06791fee-dd0c-4711-92e4-122fe234bda7 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:43.995665 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 941/1863] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:35:43 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:35:43.995919 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60c9c65-1b89-43ec-98b2-dc11baba47c7 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:44.008062 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60c9c65-1b89-43ec-98b2-dc11baba47c7 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:44.008062 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60c9c65-1b89-43ec-98b2-dc11baba47c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:44.018294 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-c60c9c65-1b89-43ec-98b2-dc11baba47c7 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:35:44.020891 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-c60c9c65-1b89-43ec-98b2-dc11baba47c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:44.034906 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 942/1864] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:35:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:35:44.038795 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489ac99b-f601-415e-91fc-2a7a0a1d8d09 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:44.038912 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489ac99b-f601-415e-91fc-2a7a0a1d8d09 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:44.039475 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489ac99b-f601-415e-91fc-2a7a0a1d8d09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:44.054753 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 923/1865] 10.222.0.66 () {60 vars in 1152 bytes} [Mon Jul 18 02:35:43 2022] POST /identity/v3/projects => generated 387 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:35:44.055877 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057b0e56-b553-4cc6-a0ab-e04c4c3903b4 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:44.060277 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-489ac99b-f601-415e-91fc-2a7a0a1d8d09 None None] MFA Rules not processed for user `80b138831bc54c7b986d50092d62d4af`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:35:44.061271 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-489ac99b-f601-415e-91fc-2a7a0a1d8d09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:44.163766 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057b0e56-b553-4cc6-a0ab-e04c4c3903b4 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:44.174414 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e9486f-ecbb-445d-87f5-e5205b0adb6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:44.175508 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e9486f-ecbb-445d-87f5-e5205b0adb6f admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:44.175899 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e9486f-ecbb-445d-87f5-e5205b0adb6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:44.176782 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e9486f-ecbb-445d-87f5-e5205b0adb6f admin admin] PATH_INFO: `/v3/projects/fe155f52b80347b6be75b811ba524b08/users/07acc53c138649e89cdbeabbd3293184/roles/65feabf3b27944a8b01a86c8ac608457` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:44.185559 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 943/1866] 10.222.0.66 () {58 vars in 1018 bytes} [Mon Jul 18 02:35:44 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:35:44.187886 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489ac99b-f601-415e-91fc-2a7a0a1d8d09 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:44.195432 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e9486f-ecbb-445d-87f5-e5205b0adb6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe155f52b80347b6be75b811ba524b08, user_id=07acc53c138649e89cdbeabbd3293184, role_id=65feabf3b27944a8b01a86c8ac608457)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:44.199932 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e9486f-ecbb-445d-87f5-e5205b0adb6f admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:44.203573 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489ac99b-f601-415e-91fc-2a7a0a1d8d09 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:44.212667 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09356830-7e8d-44cb-894a-149a690941f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:44.216128 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09356830-7e8d-44cb-894a-149a690941f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:44.216653 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09356830-7e8d-44cb-894a-149a690941f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:44.217253 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09356830-7e8d-44cb-894a-149a690941f8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:44.219125 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09356830-7e8d-44cb-894a-149a690941f8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:35:44.219578 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09356830-7e8d-44cb-894a-149a690941f8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:44.222398 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09356830-7e8d-44cb-894a-149a690941f8 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:44.240751 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 944/1867] 10.222.0.66 () {58 vars in 1152 bytes} [Mon Jul 18 02:35:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:35:44.240751 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09356830-7e8d-44cb-894a-149a690941f8 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:44.252146 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09356830-7e8d-44cb-894a-149a690941f8 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:44.257787 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193d0fa9-48bc-4ade-9550-ae881ae4593d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:44.258397 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193d0fa9-48bc-4ade-9550-ae881ae4593d admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:44.258994 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193d0fa9-48bc-4ade-9550-ae881ae4593d admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:44.259291 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193d0fa9-48bc-4ade-9550-ae881ae4593d admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:44.259999 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193d0fa9-48bc-4ade-9550-ae881ae4593d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:44.262650 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193d0fa9-48bc-4ade-9550-ae881ae4593d admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:44.314114 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 924/1868] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:35:44 2022] PUT /identity/v3/projects/fe155f52b80347b6be75b811ba524b08/users/07acc53c138649e89cdbeabbd3293184/roles/65feabf3b27944a8b01a86c8ac608457 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:35:44.315554 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e9486f-ecbb-445d-87f5-e5205b0adb6f admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:44.388925 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e9486f-ecbb-445d-87f5-e5205b0adb6f admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:44.393900 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8386da8d-0269-43be-b760-bdf3c14a86f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:44.394637 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8386da8d-0269-43be-b760-bdf3c14a86f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:44.395170 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8386da8d-0269-43be-b760-bdf3c14a86f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:44.395461 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8386da8d-0269-43be-b760-bdf3c14a86f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:44.396162 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8386da8d-0269-43be-b760-bdf3c14a86f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:44.398168 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8386da8d-0269-43be-b760-bdf3c14a86f4 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:44.411516 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 945/1869] 10.222.0.66 () {60 vars in 1143 bytes} [Mon Jul 18 02:35:44 2022] POST /identity/v3/users => generated 300 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:35:44.413208 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed43813-d0ff-46fd-b499-416ac6abd5fb None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:44.413846 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed43813-d0ff-46fd-b499-416ac6abd5fb None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:44.414239 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed43813-d0ff-46fd-b499-416ac6abd5fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:44.450393 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-5ed43813-d0ff-46fd-b499-416ac6abd5fb None None] MFA Rules not processed for user `07acc53c138649e89cdbeabbd3293184`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:35:44.451752 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-5ed43813-d0ff-46fd-b499-416ac6abd5fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:44.469946 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:35:44.513493 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 925/1870] 10.222.0.66 () {60 vars in 1152 bytes} [Mon Jul 18 02:35:44 2022] POST /identity/v3/projects => generated 385 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:35:44.514862 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8386da8d-0269-43be-b760-bdf3c14a86f4 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:44.518253 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-8386da8d-0269-43be-b760-bdf3c14a86f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:44.543679 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 946/1871] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:35:44 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:35:44.544760 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed43813-d0ff-46fd-b499-416ac6abd5fb None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:44.556299 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed43813-d0ff-46fd-b499-416ac6abd5fb None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:44.611976 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8386da8d-0269-43be-b760-bdf3c14a86f4 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:44.617867 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fc3dc5-4da9-4303-a457-863afae7c61e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80b138831bc54c7b986d50092d62d4af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:44.618548 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fc3dc5-4da9-4303-a457-863afae7c61e service nova] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:44.618829 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fc3dc5-4da9-4303-a457-863afae7c61e service nova] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:44.619119 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fc3dc5-4da9-4303-a457-863afae7c61e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:44.626830 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90743d63-cb41-410b-9feb-3de7d877e17a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:44.627720 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90743d63-cb41-410b-9feb-3de7d877e17a admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:44.628096 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90743d63-cb41-410b-9feb-3de7d877e17a admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:44.628449 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90743d63-cb41-410b-9feb-3de7d877e17a admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:44.629300 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90743d63-cb41-410b-9feb-3de7d877e17a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:44.631035 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90743d63-cb41-410b-9feb-3de7d877e17a admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:44.637468 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fc3dc5-4da9-4303-a457-863afae7c61e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:44.640815 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fc3dc5-4da9-4303-a457-863afae7c61e service nova] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:44.646389 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 947/1872] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:35:44 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:35:44.646389 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90743d63-cb41-410b-9feb-3de7d877e17a admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:44.653332 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 926/1873] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:35:44 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:35:44.654318 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fc3dc5-4da9-4303-a457-863afae7c61e service nova] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:44.654555 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90743d63-cb41-410b-9feb-3de7d877e17a admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:44.659750 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc431843-250a-4a49-bf15-4a2fa464a18b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:44.660505 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc431843-250a-4a49-bf15-4a2fa464a18b admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:44.661138 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc431843-250a-4a49-bf15-4a2fa464a18b admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:44.661402 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc431843-250a-4a49-bf15-4a2fa464a18b admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:44.662244 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc431843-250a-4a49-bf15-4a2fa464a18b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:44.664810 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc431843-250a-4a49-bf15-4a2fa464a18b admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:44.671364 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fc3dc5-4da9-4303-a457-863afae7c61e service nova] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:44.674523 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164cb3c7-1911-4e29-8093-e745415c9a48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:44.675379 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164cb3c7-1911-4e29-8093-e745415c9a48 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:44.675666 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164cb3c7-1911-4e29-8093-e745415c9a48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:44.676002 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164cb3c7-1911-4e29-8093-e745415c9a48 admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:44.676709 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164cb3c7-1911-4e29-8093-e745415c9a48 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:44.678879 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164cb3c7-1911-4e29-8093-e745415c9a48 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:44.788703 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 948/1874] 10.222.0.66 () {60 vars in 1143 bytes} [Mon Jul 18 02:35:44 2022] POST /identity/v3/users => generated 293 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:35:44.788703 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc431843-250a-4a49-bf15-4a2fa464a18b admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:44.798582 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc431843-250a-4a49-bf15-4a2fa464a18b admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:44.803201 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 927/1875] 10.222.0.66 () {60 vars in 1152 bytes} [Mon Jul 18 02:35:44 2022] POST /identity/v3/projects => generated 367 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:35:44.806227 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164cb3c7-1911-4e29-8093-e745415c9a48 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:44.861099 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b794e633-6243-4b2a-b164-55ec7bda09fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:44.862234 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b794e633-6243-4b2a-b164-55ec7bda09fa admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:44.862584 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b794e633-6243-4b2a-b164-55ec7bda09fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:44.862951 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b794e633-6243-4b2a-b164-55ec7bda09fa admin admin] PATH_INFO: `/v3/projects/444b3132a7014b32abe735e5c4944941/users/dbbb06a55c144b82bd4761edddf4b868/roles/d4aa3324f0c84887b41d0dc4acf7f30e` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:44.871222 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164cb3c7-1911-4e29-8093-e745415c9a48 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:44.876390 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d06c54b-3159-4997-9993-09440abf3f7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:44.878430 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d06c54b-3159-4997-9993-09440abf3f7b admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:44.878430 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d06c54b-3159-4997-9993-09440abf3f7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:44.878430 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d06c54b-3159-4997-9993-09440abf3f7b admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:44.878813 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d06c54b-3159-4997-9993-09440abf3f7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:44.880583 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d06c54b-3159-4997-9993-09440abf3f7b admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:44.885664 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b794e633-6243-4b2a-b164-55ec7bda09fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=444b3132a7014b32abe735e5c4944941, user_id=dbbb06a55c144b82bd4761edddf4b868, role_id=d4aa3324f0c84887b41d0dc4acf7f30e)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:44.888513 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b794e633-6243-4b2a-b164-55ec7bda09fa admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:44.892045 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 928/1876] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:35:44 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:35:44.893036 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d06c54b-3159-4997-9993-09440abf3f7b admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:44.907182 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d06c54b-3159-4997-9993-09440abf3f7b admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:44.913983 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814890a4-f580-4e01-9728-dbb0fd5096f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:44.914796 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814890a4-f580-4e01-9728-dbb0fd5096f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:44.915176 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814890a4-f580-4e01-9728-dbb0fd5096f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:44.915565 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814890a4-f580-4e01-9728-dbb0fd5096f0 admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:44.916372 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814890a4-f580-4e01-9728-dbb0fd5096f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:44.919229 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814890a4-f580-4e01-9728-dbb0fd5096f0 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:44.966330 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 949/1877] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:35:44 2022] PUT /identity/v3/projects/444b3132a7014b32abe735e5c4944941/users/dbbb06a55c144b82bd4761edddf4b868/roles/d4aa3324f0c84887b41d0dc4acf7f30e => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:35:44.970153 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b794e633-6243-4b2a-b164-55ec7bda09fa admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:45.032252 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b794e633-6243-4b2a-b164-55ec7bda09fa admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:45.037121 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad985bb-aca7-403d-92c6-012f251e89bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:45.037809 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad985bb-aca7-403d-92c6-012f251e89bf admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:45.038116 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad985bb-aca7-403d-92c6-012f251e89bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:45.038495 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad985bb-aca7-403d-92c6-012f251e89bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:45.039198 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad985bb-aca7-403d-92c6-012f251e89bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:45.040813 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad985bb-aca7-403d-92c6-012f251e89bf admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:45.055864 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 950/1878] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:35:44 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:35:45.055864 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad985bb-aca7-403d-92c6-012f251e89bf admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:45.063904 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad985bb-aca7-403d-92c6-012f251e89bf admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:45.068724 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab09add-8f7c-48a9-a118-9590660da80f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:45.069390 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab09add-8f7c-48a9-a118-9590660da80f admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:45.069680 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab09add-8f7c-48a9-a118-9590660da80f admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:45.069959 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab09add-8f7c-48a9-a118-9590660da80f admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:45.070713 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab09add-8f7c-48a9-a118-9590660da80f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:45.083486 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 929/1879] 10.222.0.66 () {60 vars in 1143 bytes} [Mon Jul 18 02:35:44 2022] POST /identity/v3/users => generated 290 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:35:45.084532 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814890a4-f580-4e01-9728-dbb0fd5096f0 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:45.096432 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab09add-8f7c-48a9-a118-9590660da80f admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:45.119869 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814890a4-f580-4e01-9728-dbb0fd5096f0 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:45.123302 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 951/1880] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:35:45 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:35:45.124442 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab09add-8f7c-48a9-a118-9590660da80f admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:45.133314 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c646f582-ba16-45f8-a62f-b2b7fd296ad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:45.140395 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c646f582-ba16-45f8-a62f-b2b7fd296ad0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:45.140903 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c646f582-ba16-45f8-a62f-b2b7fd296ad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:45.141398 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c646f582-ba16-45f8-a62f-b2b7fd296ad0 admin admin] PATH_INFO: `/v3/projects/24f3be6cd1a04a6ea79bed496d7ab16f/users/2d5439ca1b5349fa9d75729c5fa32a79/roles/65feabf3b27944a8b01a86c8ac608457` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:45.148779 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab09add-8f7c-48a9-a118-9590660da80f admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:45.154914 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d5ca58-36b8-47a1-94df-886f74557d02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:45.155612 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d5ca58-36b8-47a1-94df-886f74557d02 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:45.156491 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d5ca58-36b8-47a1-94df-886f74557d02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:45.157369 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d5ca58-36b8-47a1-94df-886f74557d02 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:45.158176 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d5ca58-36b8-47a1-94df-886f74557d02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:45.159981 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d5ca58-36b8-47a1-94df-886f74557d02 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:45.163689 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c646f582-ba16-45f8-a62f-b2b7fd296ad0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=24f3be6cd1a04a6ea79bed496d7ab16f, user_id=2d5439ca1b5349fa9d75729c5fa32a79, role_id=65feabf3b27944a8b01a86c8ac608457)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:45.168143 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c646f582-ba16-45f8-a62f-b2b7fd296ad0 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:45.173524 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 952/1881] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:35:45 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:35:45.174907 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d5ca58-36b8-47a1-94df-886f74557d02 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:45.187623 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d5ca58-36b8-47a1-94df-886f74557d02 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:45.192890 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f89a90c-49f5-472e-b927-a72d80413958 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:45.193800 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f89a90c-49f5-472e-b927-a72d80413958 admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:45.194110 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f89a90c-49f5-472e-b927-a72d80413958 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:45.194389 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f89a90c-49f5-472e-b927-a72d80413958 admin admin] PATH_INFO: `/v3/projects/444b3132a7014b32abe735e5c4944941/users/dbbb06a55c144b82bd4761edddf4b868/roles/d4aa3324f0c84887b41d0dc4acf7f30e` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:45.196813 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f89a90c-49f5-472e-b927-a72d80413958 admin admin] RBAC: Authorizing `identity:create_grant(project_id=444b3132a7014b32abe735e5c4944941, user_id=dbbb06a55c144b82bd4761edddf4b868, role_id=d4aa3324f0c84887b41d0dc4acf7f30e)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:45.199680 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f89a90c-49f5-472e-b927-a72d80413958 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:45.222275 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 953/1882] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:35:45 2022] PUT /identity/v3/projects/444b3132a7014b32abe735e5c4944941/users/dbbb06a55c144b82bd4761edddf4b868/roles/d4aa3324f0c84887b41d0dc4acf7f30e => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:35:45.222275 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f89a90c-49f5-472e-b927-a72d80413958 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:45.264138 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 930/1883] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:35:45 2022] PUT /identity/v3/projects/24f3be6cd1a04a6ea79bed496d7ab16f/users/2d5439ca1b5349fa9d75729c5fa32a79/roles/65feabf3b27944a8b01a86c8ac608457 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:35:45.277116 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c655037-f040-4c46-997a-2a4ed1d39d66 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:45.279506 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c655037-f040-4c46-997a-2a4ed1d39d66 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:45.280748 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c655037-f040-4c46-997a-2a4ed1d39d66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:45.324131 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-5c655037-f040-4c46-997a-2a4ed1d39d66 None None] MFA Rules not processed for user `dbbb06a55c144b82bd4761edddf4b868`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:35:45.326394 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-5c655037-f040-4c46-997a-2a4ed1d39d66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:45.340582 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f89a90c-49f5-472e-b927-a72d80413958 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:45.469571 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e9a199-29c5-49ce-8bf5-808e77e12ff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:45.470702 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e9a199-29c5-49ce-8bf5-808e77e12ff3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:45.471091 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e9a199-29c5-49ce-8bf5-808e77e12ff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:45.472054 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e9a199-29c5-49ce-8bf5-808e77e12ff3 admin admin] PATH_INFO: `/v3/projects/6b23c3e795b84ab7b749da296a6f4c3f/users/f374e81e77f24d48aea52effd0164e7c/roles/d4aa3324f0c84887b41d0dc4acf7f30e` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:45.494955 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e9a199-29c5-49ce-8bf5-808e77e12ff3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b23c3e795b84ab7b749da296a6f4c3f, user_id=f374e81e77f24d48aea52effd0164e7c, role_id=d4aa3324f0c84887b41d0dc4acf7f30e)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:45.496977 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 931/1884] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:35:45 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:35:45.498630 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ac6fb4-3d3d-4e65-b670-01f037a46a75 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:45.498727 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ac6fb4-3d3d-4e65-b670-01f037a46a75 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:45.499013 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ac6fb4-3d3d-4e65-b670-01f037a46a75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:45.499261 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e9a199-29c5-49ce-8bf5-808e77e12ff3 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:45.533456 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-67ac6fb4-3d3d-4e65-b670-01f037a46a75 None None] MFA Rules not processed for user `2d5439ca1b5349fa9d75729c5fa32a79`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:35:45.534781 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-67ac6fb4-3d3d-4e65-b670-01f037a46a75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:45.591547 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 954/1885] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:35:45 2022] PUT /identity/v3/projects/6b23c3e795b84ab7b749da296a6f4c3f/users/f374e81e77f24d48aea52effd0164e7c/roles/d4aa3324f0c84887b41d0dc4acf7f30e => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:35:45.591547 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e9a199-29c5-49ce-8bf5-808e77e12ff3 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:45.593274 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-13e9a199-29c5-49ce-8bf5-808e77e12ff3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:45.630766 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 932/1886] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:35:45 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:35:45.631773 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ac6fb4-3d3d-4e65-b670-01f037a46a75 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:45.648983 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ac6fb4-3d3d-4e65-b670-01f037a46a75 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:45.667943 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e9a199-29c5-49ce-8bf5-808e77e12ff3 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:45.686250 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00d4eb6-e8ac-4dfd-95e0-a032d88352a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd49388facdf446885e55b45ef91f01e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:45.686250 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00d4eb6-e8ac-4dfd-95e0-a032d88352a7 service cinder] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:45.686250 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00d4eb6-e8ac-4dfd-95e0-a032d88352a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:45.686250 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00d4eb6-e8ac-4dfd-95e0-a032d88352a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:45.764822 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a12d4fc-8d1f-4bc3-87e3-447b63801291 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:45.765945 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a12d4fc-8d1f-4bc3-87e3-447b63801291 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:45.766283 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a12d4fc-8d1f-4bc3-87e3-447b63801291 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:45.766567 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a12d4fc-8d1f-4bc3-87e3-447b63801291 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:45.767310 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00d4eb6-e8ac-4dfd-95e0-a032d88352a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:45.769114 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00d4eb6-e8ac-4dfd-95e0-a032d88352a7 service cinder] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:45.773197 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a12d4fc-8d1f-4bc3-87e3-447b63801291 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:45.776159 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a12d4fc-8d1f-4bc3-87e3-447b63801291 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:45.782006 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 955/1887] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:35:45 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:35:45.783424 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00d4eb6-e8ac-4dfd-95e0-a032d88352a7 service cinder] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:45.790143 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 933/1888] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:35:45 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:35:45.791827 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a12d4fc-8d1f-4bc3-87e3-447b63801291 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:45.796767 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00d4eb6-e8ac-4dfd-95e0-a032d88352a7 service cinder] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:45.802574 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9213c57-cf14-4168-9910-8bb5770af341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:45.803392 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9213c57-cf14-4168-9910-8bb5770af341 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:45.803914 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9213c57-cf14-4168-9910-8bb5770af341 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:45.804808 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9213c57-cf14-4168-9910-8bb5770af341 admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:45.805015 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9213c57-cf14-4168-9910-8bb5770af341 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:45.805616 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a12d4fc-8d1f-4bc3-87e3-447b63801291 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:45.807392 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9213c57-cf14-4168-9910-8bb5770af341 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:45.815667 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1a3114-4090-4038-8da5-b996a3821730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:45.816665 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1a3114-4090-4038-8da5-b996a3821730 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:45.817202 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1a3114-4090-4038-8da5-b996a3821730 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:45.817566 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1a3114-4090-4038-8da5-b996a3821730 admin admin] PATH_INFO: `/v3/projects/6b23c3e795b84ab7b749da296a6f4c3f/users/f374e81e77f24d48aea52effd0164e7c/roles/d4aa3324f0c84887b41d0dc4acf7f30e` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:45.820395 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1a3114-4090-4038-8da5-b996a3821730 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b23c3e795b84ab7b749da296a6f4c3f, user_id=f374e81e77f24d48aea52effd0164e7c, role_id=d4aa3324f0c84887b41d0dc4acf7f30e)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:45.824031 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1a3114-4090-4038-8da5-b996a3821730 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:45.826997 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:35:45.854586 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 934/1889] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:35:45 2022] PUT /identity/v3/projects/6b23c3e795b84ab7b749da296a6f4c3f/users/f374e81e77f24d48aea52effd0164e7c/roles/d4aa3324f0c84887b41d0dc4acf7f30e => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:35:45.860072 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab67cc87-4cf3-403d-a1d5-4daabebfa3f3 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:45.861741 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:35:45.863065 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab67cc87-4cf3-403d-a1d5-4daabebfa3f3 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:45.863884 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab67cc87-4cf3-403d-a1d5-4daabebfa3f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:45.893259 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-ab67cc87-4cf3-403d-a1d5-4daabebfa3f3 None None] MFA Rules not processed for user `f374e81e77f24d48aea52effd0164e7c`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:35:45.894572 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-ab67cc87-4cf3-403d-a1d5-4daabebfa3f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:45.914237 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 956/1890] 10.222.0.66 () {60 vars in 1152 bytes} [Mon Jul 18 02:35:45 2022] POST /identity/v3/projects => generated 383 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:35:45.919143 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9213c57-cf14-4168-9910-8bb5770af341 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:45.986995 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 935/1891] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:35:45 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:35:45.990295 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9213c57-cf14-4168-9910-8bb5770af341 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:45.995267 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ca91c3-99d6-4c87-bfff-d4c84e7dc768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:45.996102 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ca91c3-99d6-4c87-bfff-d4c84e7dc768 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:45.996458 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ca91c3-99d6-4c87-bfff-d4c84e7dc768 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:45.996823 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ca91c3-99d6-4c87-bfff-d4c84e7dc768 admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:45.997601 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ca91c3-99d6-4c87-bfff-d4c84e7dc768 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:45.999665 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ca91c3-99d6-4c87-bfff-d4c84e7dc768 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:46.006499 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab67cc87-4cf3-403d-a1d5-4daabebfa3f3 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:46.008252 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-ab67cc87-4cf3-403d-a1d5-4daabebfa3f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:46.067525 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab67cc87-4cf3-403d-a1d5-4daabebfa3f3 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:46.117378 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 957/1892] 10.222.0.66 () {60 vars in 1143 bytes} [Mon Jul 18 02:35:45 2022] POST /identity/v3/users => generated 298 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:35:46.120954 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5372cf41-fe76-4012-86c5-9a0d5f9eac11 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd49388facdf446885e55b45ef91f01e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:46.121674 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5372cf41-fe76-4012-86c5-9a0d5f9eac11 service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:46.121788 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ca91c3-99d6-4c87-bfff-d4c84e7dc768 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:46.121958 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5372cf41-fe76-4012-86c5-9a0d5f9eac11 service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:46.122254 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5372cf41-fe76-4012-86c5-9a0d5f9eac11 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:46.135528 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ca91c3-99d6-4c87-bfff-d4c84e7dc768 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:46.145073 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cc68e8-6665-4f88-8b52-c8f9762fb49d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:46.146819 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cc68e8-6665-4f88-8b52-c8f9762fb49d admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:46.147240 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cc68e8-6665-4f88-8b52-c8f9762fb49d admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:46.147587 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cc68e8-6665-4f88-8b52-c8f9762fb49d admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:46.148570 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cc68e8-6665-4f88-8b52-c8f9762fb49d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:46.150705 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cc68e8-6665-4f88-8b52-c8f9762fb49d admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:46.165945 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 958/1893] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:35:46 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 02:35:46.171085 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cc68e8-6665-4f88-8b52-c8f9762fb49d admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:46.183804 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cc68e8-6665-4f88-8b52-c8f9762fb49d admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:46.193067 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ed3d9e-b72e-45a5-a8b1-20a0dd4a87af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:46.194189 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ed3d9e-b72e-45a5-a8b1-20a0dd4a87af admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:46.194610 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ed3d9e-b72e-45a5-a8b1-20a0dd4a87af admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:46.194969 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ed3d9e-b72e-45a5-a8b1-20a0dd4a87af admin admin] PATH_INFO: `/v3/projects/af185a33c0cb490eaa6fca1912d794f0/users/49b60e818ad546df8353723ac6d8464a/roles/d4aa3324f0c84887b41d0dc4acf7f30e` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:46.198271 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5372cf41-fe76-4012-86c5-9a0d5f9eac11 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:46.199827 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5372cf41-fe76-4012-86c5-9a0d5f9eac11 service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:46.211246 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 936/1894] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:35:45 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 02:35:46.220618 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ed3d9e-b72e-45a5-a8b1-20a0dd4a87af admin admin] RBAC: Authorizing `identity:create_grant(project_id=af185a33c0cb490eaa6fca1912d794f0, user_id=49b60e818ad546df8353723ac6d8464a, role_id=d4aa3324f0c84887b41d0dc4acf7f30e)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:46.223502 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ed3d9e-b72e-45a5-a8b1-20a0dd4a87af admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:46.302898 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 959/1895] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:35:46 2022] PUT /identity/v3/projects/af185a33c0cb490eaa6fca1912d794f0/users/49b60e818ad546df8353723ac6d8464a/roles/d4aa3324f0c84887b41d0dc4acf7f30e => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:35:46.307451 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5372cf41-fe76-4012-86c5-9a0d5f9eac11 service cinder] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:46.318204 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ed3d9e-b72e-45a5-a8b1-20a0dd4a87af admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:46.319566 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5372cf41-fe76-4012-86c5-9a0d5f9eac11 service cinder] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:46.380944 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ed3d9e-b72e-45a5-a8b1-20a0dd4a87af admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:46.382300 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebe22ab-9a2a-4562-8346-a98e66217c77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:46.383193 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebe22ab-9a2a-4562-8346-a98e66217c77 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:46.383552 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebe22ab-9a2a-4562-8346-a98e66217c77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:46.383897 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebe22ab-9a2a-4562-8346-a98e66217c77 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:46.384697 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebe22ab-9a2a-4562-8346-a98e66217c77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:46.385075 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d3733a-aab6-4795-8f90-3fcd0e401fec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd49388facdf446885e55b45ef91f01e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:46.386346 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d3733a-aab6-4795-8f90-3fcd0e401fec service cinder] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:46.386726 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d3733a-aab6-4795-8f90-3fcd0e401fec service cinder] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:46.387223 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebe22ab-9a2a-4562-8346-a98e66217c77 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:46.387342 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d3733a-aab6-4795-8f90-3fcd0e401fec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:46.398168 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 937/1896] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:35:46 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:35:46.403009 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebe22ab-9a2a-4562-8346-a98e66217c77 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:46.413608 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebe22ab-9a2a-4562-8346-a98e66217c77 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:46.419483 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a783753b-4a7a-422f-a600-9cb9ae185331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:46.420408 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a783753b-4a7a-422f-a600-9cb9ae185331 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:46.420695 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a783753b-4a7a-422f-a600-9cb9ae185331 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:46.420976 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a783753b-4a7a-422f-a600-9cb9ae185331 admin admin] PATH_INFO: `/v3/projects/af185a33c0cb490eaa6fca1912d794f0/users/49b60e818ad546df8353723ac6d8464a/roles/d4aa3324f0c84887b41d0dc4acf7f30e` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:46.423232 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a783753b-4a7a-422f-a600-9cb9ae185331 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af185a33c0cb490eaa6fca1912d794f0, user_id=49b60e818ad546df8353723ac6d8464a, role_id=d4aa3324f0c84887b41d0dc4acf7f30e)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:46.425984 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a783753b-4a7a-422f-a600-9cb9ae185331 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:46.438161 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 938/1897] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:35:46 2022] PUT /identity/v3/projects/af185a33c0cb490eaa6fca1912d794f0/users/49b60e818ad546df8353723ac6d8464a/roles/d4aa3324f0c84887b41d0dc4acf7f30e => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 02:35:46.452456 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a2039b-d140-4ccb-908b-ddf7dc7d754a None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:46.452751 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a2039b-d140-4ccb-908b-ddf7dc7d754a None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:46.453160 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a2039b-d140-4ccb-908b-ddf7dc7d754a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:46.457337 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d3733a-aab6-4795-8f90-3fcd0e401fec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:46.458860 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d3733a-aab6-4795-8f90-3fcd0e401fec service cinder] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:46.469302 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 960/1898] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:35:46 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:35:46.487348 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-a6a2039b-d140-4ccb-908b-ddf7dc7d754a None None] MFA Rules not processed for user `49b60e818ad546df8353723ac6d8464a`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:35:46.488637 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-a6a2039b-d140-4ccb-908b-ddf7dc7d754a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:46.558155 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 939/1899] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:35:46 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 02:35:46.904728 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d3733a-aab6-4795-8f90-3fcd0e401fec service cinder] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:46.913081 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d3733a-aab6-4795-8f90-3fcd0e401fec service cinder] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:46.969629 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a92667-3a96-4605-bc2d-47ed125592aa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd49388facdf446885e55b45ef91f01e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:46.970488 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a92667-3a96-4605-bc2d-47ed125592aa service cinder] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:46.970718 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a92667-3a96-4605-bc2d-47ed125592aa service cinder] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:46.970991 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a92667-3a96-4605-bc2d-47ed125592aa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:46.981727 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a92667-3a96-4605-bc2d-47ed125592aa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:46.982994 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a92667-3a96-4605-bc2d-47ed125592aa service cinder] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:46.994260 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 961/1900] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:35:46 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:35:47.085811 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a2039b-d140-4ccb-908b-ddf7dc7d754a None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:47.150263 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a2039b-d140-4ccb-908b-ddf7dc7d754a None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:47.156398 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ea5eba-6cab-4d64-ab4e-2255b75aa777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:47.157272 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ea5eba-6cab-4d64-ab4e-2255b75aa777 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:47.157575 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ea5eba-6cab-4d64-ab4e-2255b75aa777 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:47.157837 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ea5eba-6cab-4d64-ab4e-2255b75aa777 admin admin] PATH_INFO: `/v3/users/2d5439ca1b5349fa9d75729c5fa32a79` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:47.159063 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ea5eba-6cab-4d64-ab4e-2255b75aa777 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d5439ca1b5349fa9d75729c5fa32a79)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:47.161032 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ea5eba-6cab-4d64-ab4e-2255b75aa777 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:47.200732 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a92667-3a96-4605-bc2d-47ed125592aa service cinder] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:47.206062 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=82,keystone:DELETE=14,keystone:INSERT=4 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:35:47.210941 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a92667-3a96-4605-bc2d-47ed125592aa service cinder] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:47.216235 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc3677d-a809-4fff-901d-5a9255b63347 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd49388facdf446885e55b45ef91f01e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:47.218430 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc3677d-a809-4fff-901d-5a9255b63347 service cinder] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:47.218430 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc3677d-a809-4fff-901d-5a9255b63347 service cinder] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:47.218430 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc3677d-a809-4fff-901d-5a9255b63347 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:47.219919 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-cfc3677d-a809-4fff-901d-5a9255b63347 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:47.283467 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc3677d-a809-4fff-901d-5a9255b63347 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:47.285151 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc3677d-a809-4fff-901d-5a9255b63347 service cinder] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:47.294601 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 962/1901] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:35:47 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:35:47.362276 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-d4ea5eba-6cab-4d64-ab4e-2255b75aa777 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d5439ca1b5349fa9d75729c5fa32a79'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:47.388987 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:35:47.451521 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-d4ea5eba-6cab-4d64-ab4e-2255b75aa777 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d5439ca1b5349fa9d75729c5fa32a79'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:47.452140 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-d4ea5eba-6cab-4d64-ab4e-2255b75aa777 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d5439ca1b5349fa9d75729c5fa32a79'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:47.465370 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-d4ea5eba-6cab-4d64-ab4e-2255b75aa777 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d5439ca1b5349fa9d75729c5fa32a79'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:47.466063 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-d4ea5eba-6cab-4d64-ab4e-2255b75aa777 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d5439ca1b5349fa9d75729c5fa32a79'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:47.478021 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 940/1902] 10.222.0.66 () {58 vars in 1223 bytes} [Mon Jul 18 02:35:47 2022] DELETE /identity/v3/users/2d5439ca1b5349fa9d75729c5fa32a79 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:35:47.483230 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be835764-fca6-4f3e-ac83-3fcfd03d321b None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:47.483684 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be835764-fca6-4f3e-ac83-3fcfd03d321b None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:47.483817 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be835764-fca6-4f3e-ac83-3fcfd03d321b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:47.501073 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-be835764-fca6-4f3e-ac83-3fcfd03d321b None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:35:47.513574 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-be835764-fca6-4f3e-ac83-3fcfd03d321b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:47.584098 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 963/1903] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:35:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:35:47.595774 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ea5eba-6cab-4d64-ab4e-2255b75aa777 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:47.600737 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-d4ea5eba-6cab-4d64-ab4e-2255b75aa777 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:47.662680 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ea5eba-6cab-4d64-ab4e-2255b75aa777 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:47.666276 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0067cae7-58b7-4868-95e3-7ed8caaf2da4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eaf19a259c2741cea375a38763e33cdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:47.667060 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0067cae7-58b7-4868-95e3-7ed8caaf2da4 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:47.667426 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0067cae7-58b7-4868-95e3-7ed8caaf2da4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:47.667726 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0067cae7-58b7-4868-95e3-7ed8caaf2da4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:47.684614 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0067cae7-58b7-4868-95e3-7ed8caaf2da4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:47.684614 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0067cae7-58b7-4868-95e3-7ed8caaf2da4 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:47.691064 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 941/1904] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:35:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 02:35:48.262577 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be835764-fca6-4f3e-ac83-3fcfd03d321b None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:48.265220 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-be835764-fca6-4f3e-ac83-3fcfd03d321b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:48.279466 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be835764-fca6-4f3e-ac83-3fcfd03d321b None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:48.284475 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8129cda1-0e42-41c5-ad51-a351a019a56c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:48.285349 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8129cda1-0e42-41c5-ad51-a351a019a56c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:48.285642 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8129cda1-0e42-41c5-ad51-a351a019a56c admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:48.285917 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8129cda1-0e42-41c5-ad51-a351a019a56c admin admin] PATH_INFO: `/v3/projects/24f3be6cd1a04a6ea79bed496d7ab16f` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:48.287271 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8129cda1-0e42-41c5-ad51-a351a019a56c admin admin] RBAC: Authorizing `identity:delete_project(project_id=24f3be6cd1a04a6ea79bed496d7ab16f)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:48.289310 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8129cda1-0e42-41c5-ad51-a351a019a56c admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:48.421292 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-8129cda1-0e42-41c5-ad51-a351a019a56c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24f3be6cd1a04a6ea79bed496d7ab16f', 'request_id': 'req-8129cda1-0e42-41c5-ad51-a351a019a56c'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:48.427041 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-8129cda1-0e42-41c5-ad51-a351a019a56c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24f3be6cd1a04a6ea79bed496d7ab16f', 'request_id': 'req-8129cda1-0e42-41c5-ad51-a351a019a56c'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:48.509132 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-8129cda1-0e42-41c5-ad51-a351a019a56c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24f3be6cd1a04a6ea79bed496d7ab16f', 'request_id': 'req-8129cda1-0e42-41c5-ad51-a351a019a56c'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:48.522769 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-8129cda1-0e42-41c5-ad51-a351a019a56c admin admin] The token cache is being invalidate because project 24f3be6cd1a04a6ea79bed496d7ab16f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 02:35:48.523065 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-8129cda1-0e42-41c5-ad51-a351a019a56c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:48.523732 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-8129cda1-0e42-41c5-ad51-a351a019a56c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:48.525372 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 964/1905] 10.222.0.66 () {58 vars in 1232 bytes} [Mon Jul 18 02:35:48 2022] DELETE /identity/v3/projects/24f3be6cd1a04a6ea79bed496d7ab16f => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:35:48.530226 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0067cae7-58b7-4868-95e3-7ed8caaf2da4 service neutron] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:48.540456 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0067cae7-58b7-4868-95e3-7ed8caaf2da4 service neutron] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:48.599130 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f76c02e-5358-4102-aed1-4fe368a0100b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:48.600140 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f76c02e-5358-4102-aed1-4fe368a0100b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:48.600391 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f76c02e-5358-4102-aed1-4fe368a0100b admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:48.600697 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f76c02e-5358-4102-aed1-4fe368a0100b admin admin] PATH_INFO: `/v3/users/49b60e818ad546df8353723ac6d8464a` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:48.601926 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f76c02e-5358-4102-aed1-4fe368a0100b admin admin] RBAC: Authorizing `identity:delete_user(user_id=49b60e818ad546df8353723ac6d8464a)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:48.603897 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f76c02e-5358-4102-aed1-4fe368a0100b admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:48.867152 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-0f76c02e-5358-4102-aed1-4fe368a0100b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49b60e818ad546df8353723ac6d8464a'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:49.017996 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-0f76c02e-5358-4102-aed1-4fe368a0100b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49b60e818ad546df8353723ac6d8464a'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:49.018689 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-0f76c02e-5358-4102-aed1-4fe368a0100b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49b60e818ad546df8353723ac6d8464a'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:49.034322 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-0f76c02e-5358-4102-aed1-4fe368a0100b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49b60e818ad546df8353723ac6d8464a'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:49.035212 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-0f76c02e-5358-4102-aed1-4fe368a0100b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49b60e818ad546df8353723ac6d8464a'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:49.052491 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 942/1906] 10.222.0.66 () {58 vars in 1223 bytes} [Mon Jul 18 02:35:48 2022] DELETE /identity/v3/users/49b60e818ad546df8353723ac6d8464a => generated 0 bytes in 523 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:35:49.312314 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:35:49.617595 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:35:49.647164 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8129cda1-0e42-41c5-ad51-a351a019a56c admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:49.648894 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-8129cda1-0e42-41c5-ad51-a351a019a56c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:49.665029 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8129cda1-0e42-41c5-ad51-a351a019a56c admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:49.723573 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bef96d9-5744-4aec-bd17-06b0a9a034d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:49.724366 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bef96d9-5744-4aec-bd17-06b0a9a034d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:49.724655 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bef96d9-5744-4aec-bd17-06b0a9a034d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:49.724960 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bef96d9-5744-4aec-bd17-06b0a9a034d6 admin admin] PATH_INFO: `/v3/projects/af185a33c0cb490eaa6fca1912d794f0` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:49.726208 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bef96d9-5744-4aec-bd17-06b0a9a034d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=af185a33c0cb490eaa6fca1912d794f0)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:49.728172 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bef96d9-5744-4aec-bd17-06b0a9a034d6 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:49.839679 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-3bef96d9-5744-4aec-bd17-06b0a9a034d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af185a33c0cb490eaa6fca1912d794f0', 'request_id': 'req-3bef96d9-5744-4aec-bd17-06b0a9a034d6'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:49.845641 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-3bef96d9-5744-4aec-bd17-06b0a9a034d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af185a33c0cb490eaa6fca1912d794f0', 'request_id': 'req-3bef96d9-5744-4aec-bd17-06b0a9a034d6'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:49.958130 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-3bef96d9-5744-4aec-bd17-06b0a9a034d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af185a33c0cb490eaa6fca1912d794f0', 'request_id': 'req-3bef96d9-5744-4aec-bd17-06b0a9a034d6'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:49.971625 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-3bef96d9-5744-4aec-bd17-06b0a9a034d6 admin admin] The token cache is being invalidate because project af185a33c0cb490eaa6fca1912d794f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 02:35:49.971917 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-3bef96d9-5744-4aec-bd17-06b0a9a034d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:49.972807 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-3bef96d9-5744-4aec-bd17-06b0a9a034d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:49.974778 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 965/1907] 10.222.0.66 () {58 vars in 1232 bytes} [Mon Jul 18 02:35:49 2022] DELETE /identity/v3/projects/af185a33c0cb490eaa6fca1912d794f0 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:35:49.980150 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea04a355-6704-4103-810d-8d66fec296aa None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:49.980494 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea04a355-6704-4103-810d-8d66fec296aa None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:49.980808 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea04a355-6704-4103-810d-8d66fec296aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:50.000226 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-ea04a355-6704-4103-810d-8d66fec296aa None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:35:50.001748 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-ea04a355-6704-4103-810d-8d66fec296aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:50.070180 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 943/1908] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:35:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:35:50.078638 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d0162d-3dd9-4744-9c83-adfd0f5ecc05 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:50.079013 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d0162d-3dd9-4744-9c83-adfd0f5ecc05 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:50.079526 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d0162d-3dd9-4744-9c83-adfd0f5ecc05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:50.101849 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-11d0162d-3dd9-4744-9c83-adfd0f5ecc05 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:35:50.103543 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-11d0162d-3dd9-4744-9c83-adfd0f5ecc05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:50.112275 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 966/1909] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:35:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:35:50.118221 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea04a355-6704-4103-810d-8d66fec296aa None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:50.128261 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea04a355-6704-4103-810d-8d66fec296aa None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:50.133070 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef668a45-5df6-4fa3-a8a0-b968132060ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:50.133732 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef668a45-5df6-4fa3-a8a0-b968132060ad admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:50.134013 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef668a45-5df6-4fa3-a8a0-b968132060ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:50.134318 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef668a45-5df6-4fa3-a8a0-b968132060ad admin admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:50.135041 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef668a45-5df6-4fa3-a8a0-b968132060ad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:35:50.135901 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef668a45-5df6-4fa3-a8a0-b968132060ad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:50.137530 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef668a45-5df6-4fa3-a8a0-b968132060ad admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:50.152459 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 944/1910] 10.222.0.66 () {58 vars in 1152 bytes} [Mon Jul 18 02:35:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 02:35:50.156303 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d0162d-3dd9-4744-9c83-adfd0f5ecc05 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:50.166652 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d0162d-3dd9-4744-9c83-adfd0f5ecc05 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:50.172442 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92e1253-eaef-44db-aac0-ab1c240311a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:50.173207 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92e1253-eaef-44db-aac0-ab1c240311a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:50.173669 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92e1253-eaef-44db-aac0-ab1c240311a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:50.174044 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92e1253-eaef-44db-aac0-ab1c240311a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:50.174885 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92e1253-eaef-44db-aac0-ab1c240311a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:50.176982 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92e1253-eaef-44db-aac0-ab1c240311a4 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:50.276010 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 967/1911] 10.222.0.66 () {60 vars in 1152 bytes} [Mon Jul 18 02:35:50 2022] POST /identity/v3/projects => generated 381 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:35:50.280791 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef668a45-5df6-4fa3-a8a0-b968132060ad admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:50.290768 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef668a45-5df6-4fa3-a8a0-b968132060ad admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:50.361664 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efe0a5b-e146-4e21-a9d0-f4c0484331d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:50.362450 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efe0a5b-e146-4e21-a9d0-f4c0484331d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:50.363041 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efe0a5b-e146-4e21-a9d0-f4c0484331d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:50.363380 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efe0a5b-e146-4e21-a9d0-f4c0484331d9 admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:50.364191 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efe0a5b-e146-4e21-a9d0-f4c0484331d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:50.366288 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efe0a5b-e146-4e21-a9d0-f4c0484331d9 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:50.441559 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 945/1912] 10.222.0.66 () {60 vars in 1143 bytes} [Mon Jul 18 02:35:50 2022] POST /identity/v3/users => generated 291 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:35:50.446404 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92e1253-eaef-44db-aac0-ab1c240311a4 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:50.460480 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92e1253-eaef-44db-aac0-ab1c240311a4 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:50.469910 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef703090-f84a-47f5-8b7b-11ba42aa94df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:50.470634 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef703090-f84a-47f5-8b7b-11ba42aa94df admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:50.470922 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef703090-f84a-47f5-8b7b-11ba42aa94df admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:50.471201 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef703090-f84a-47f5-8b7b-11ba42aa94df admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:50.471942 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef703090-f84a-47f5-8b7b-11ba42aa94df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:50.474375 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef703090-f84a-47f5-8b7b-11ba42aa94df admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:50.487282 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 968/1913] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:35:50 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:35:50.490926 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efe0a5b-e146-4e21-a9d0-f4c0484331d9 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:50.505883 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efe0a5b-e146-4e21-a9d0-f4c0484331d9 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:50.513135 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fb4f22-e36e-47b5-b816-c924a35f8a70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:50.514322 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fb4f22-e36e-47b5-b816-c924a35f8a70 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:50.514792 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fb4f22-e36e-47b5-b816-c924a35f8a70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:50.515488 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fb4f22-e36e-47b5-b816-c924a35f8a70 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:50.521598 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fb4f22-e36e-47b5-b816-c924a35f8a70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:50.521598 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fb4f22-e36e-47b5-b816-c924a35f8a70 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:50.534379 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 946/1914] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:35:50 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:35:50.538306 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef703090-f84a-47f5-8b7b-11ba42aa94df admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:50.564903 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef703090-f84a-47f5-8b7b-11ba42aa94df admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:50.576602 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef3bdcc-2c93-4ba9-8b4c-c8a28de0c296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:50.577653 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef3bdcc-2c93-4ba9-8b4c-c8a28de0c296 admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:50.578038 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef3bdcc-2c93-4ba9-8b4c-c8a28de0c296 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:50.578706 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef3bdcc-2c93-4ba9-8b4c-c8a28de0c296 admin admin] PATH_INFO: `/v3/projects/e893953bc0df4b7ab334aca2fc15a8e1/users/0ece93dcef974adbaaf83f8264a5a4f8/roles/65feabf3b27944a8b01a86c8ac608457` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:50.647920 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef3bdcc-2c93-4ba9-8b4c-c8a28de0c296 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e893953bc0df4b7ab334aca2fc15a8e1, user_id=0ece93dcef974adbaaf83f8264a5a4f8, role_id=65feabf3b27944a8b01a86c8ac608457)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:50.655426 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef3bdcc-2c93-4ba9-8b4c-c8a28de0c296 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:50.772200 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 969/1915] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:35:50 2022] PUT /identity/v3/projects/e893953bc0df4b7ab334aca2fc15a8e1/users/0ece93dcef974adbaaf83f8264a5a4f8/roles/65feabf3b27944a8b01a86c8ac608457 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:35:50.783599 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e3f498-62e5-4642-91d5-04696cd25ce6 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:50.784165 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e3f498-62e5-4642-91d5-04696cd25ce6 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:50.784165 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e3f498-62e5-4642-91d5-04696cd25ce6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:50.826797 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-84e3f498-62e5-4642-91d5-04696cd25ce6 None None] MFA Rules not processed for user `0ece93dcef974adbaaf83f8264a5a4f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:35:50.828028 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-84e3f498-62e5-4642-91d5-04696cd25ce6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:50.899642 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 947/1916] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:35:50 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:35:50.903927 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef3bdcc-2c93-4ba9-8b4c-c8a28de0c296 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:50.964583 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef3bdcc-2c93-4ba9-8b4c-c8a28de0c296 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:50.969429 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69919beb-2354-4983-a7e0-7d0d7f428783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:50.970124 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69919beb-2354-4983-a7e0-7d0d7f428783 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:50.970428 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69919beb-2354-4983-a7e0-7d0d7f428783 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:50.970705 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69919beb-2354-4983-a7e0-7d0d7f428783 admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:50.971395 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69919beb-2354-4983-a7e0-7d0d7f428783 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:50.973485 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69919beb-2354-4983-a7e0-7d0d7f428783 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:51.087747 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 970/1917] 10.222.0.66 () {60 vars in 1152 bytes} [Mon Jul 18 02:35:50 2022] POST /identity/v3/projects => generated 381 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:35:51.091900 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e3f498-62e5-4642-91d5-04696cd25ce6 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:51.108972 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e3f498-62e5-4642-91d5-04696cd25ce6 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:51.211181 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea8fb75-8d32-4abd-a061-ded1c39d850c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:51.216430 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea8fb75-8d32-4abd-a061-ded1c39d850c admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:51.216430 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea8fb75-8d32-4abd-a061-ded1c39d850c admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:51.216430 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea8fb75-8d32-4abd-a061-ded1c39d850c admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:51.216430 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea8fb75-8d32-4abd-a061-ded1c39d850c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:51.216430 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea8fb75-8d32-4abd-a061-ded1c39d850c admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:51.303485 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 948/1918] 10.222.0.66 () {60 vars in 1143 bytes} [Mon Jul 18 02:35:51 2022] POST /identity/v3/users => generated 297 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:35:51.307584 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69919beb-2354-4983-a7e0-7d0d7f428783 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:51.318256 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69919beb-2354-4983-a7e0-7d0d7f428783 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:51.323333 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed200d04-ce0f-4fa7-9b82-b3c6114991da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:51.324042 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed200d04-ce0f-4fa7-9b82-b3c6114991da admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:51.324333 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed200d04-ce0f-4fa7-9b82-b3c6114991da admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:51.324611 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed200d04-ce0f-4fa7-9b82-b3c6114991da admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:51.325337 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed200d04-ce0f-4fa7-9b82-b3c6114991da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:51.327005 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed200d04-ce0f-4fa7-9b82-b3c6114991da admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:51.338134 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 971/1919] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:35:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:35:51.343084 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea8fb75-8d32-4abd-a061-ded1c39d850c admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:51.353147 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea8fb75-8d32-4abd-a061-ded1c39d850c admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:51.359134 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5b4a46-2f9c-41f7-89ef-a700a95b3b1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:51.360024 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5b4a46-2f9c-41f7-89ef-a700a95b3b1c admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:51.360302 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5b4a46-2f9c-41f7-89ef-a700a95b3b1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:51.360765 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5b4a46-2f9c-41f7-89ef-a700a95b3b1c admin admin] PATH_INFO: `/v3/projects/0eb5945573434e27b9b90d5b6099fae9/users/dc5d6e29468d4e0abe5a46f1a28d1722/roles/d4aa3324f0c84887b41d0dc4acf7f30e` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:51.377658 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5b4a46-2f9c-41f7-89ef-a700a95b3b1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=0eb5945573434e27b9b90d5b6099fae9, user_id=dc5d6e29468d4e0abe5a46f1a28d1722, role_id=d4aa3324f0c84887b41d0dc4acf7f30e)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:51.380645 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5b4a46-2f9c-41f7-89ef-a700a95b3b1c admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:51.468899 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 949/1920] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:35:51 2022] PUT /identity/v3/projects/0eb5945573434e27b9b90d5b6099fae9/users/dc5d6e29468d4e0abe5a46f1a28d1722/roles/d4aa3324f0c84887b41d0dc4acf7f30e => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:35:51.472975 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed200d04-ce0f-4fa7-9b82-b3c6114991da admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:51.487573 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed200d04-ce0f-4fa7-9b82-b3c6114991da admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:51.570021 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1653a5-d1c7-4ed3-a291-2c98ca9aabbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:51.570829 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1653a5-d1c7-4ed3-a291-2c98ca9aabbd admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:51.571188 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1653a5-d1c7-4ed3-a291-2c98ca9aabbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:51.571538 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1653a5-d1c7-4ed3-a291-2c98ca9aabbd admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:51.573753 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1653a5-d1c7-4ed3-a291-2c98ca9aabbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:51.581833 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1653a5-d1c7-4ed3-a291-2c98ca9aabbd admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:51.597166 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 972/1921] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:35:51 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:35:51.607975 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5b4a46-2f9c-41f7-89ef-a700a95b3b1c admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:51.626774 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5b4a46-2f9c-41f7-89ef-a700a95b3b1c admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:51.632005 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c077b6-bf97-4759-a7a5-8432dd0cfda2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:51.632982 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c077b6-bf97-4759-a7a5-8432dd0cfda2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:51.633326 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c077b6-bf97-4759-a7a5-8432dd0cfda2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:51.634067 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c077b6-bf97-4759-a7a5-8432dd0cfda2 admin admin] PATH_INFO: `/v3/projects/0eb5945573434e27b9b90d5b6099fae9/users/dc5d6e29468d4e0abe5a46f1a28d1722/roles/d4aa3324f0c84887b41d0dc4acf7f30e` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:51.636536 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c077b6-bf97-4759-a7a5-8432dd0cfda2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0eb5945573434e27b9b90d5b6099fae9, user_id=dc5d6e29468d4e0abe5a46f1a28d1722, role_id=d4aa3324f0c84887b41d0dc4acf7f30e)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:51.639316 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c077b6-bf97-4759-a7a5-8432dd0cfda2 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:51.650651 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 950/1922] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:35:51 2022] PUT /identity/v3/projects/0eb5945573434e27b9b90d5b6099fae9/users/dc5d6e29468d4e0abe5a46f1a28d1722/roles/d4aa3324f0c84887b41d0dc4acf7f30e => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 02:35:51.662691 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7549357d-e172-437e-8a29-20d8174b8d08 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:51.663360 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7549357d-e172-437e-8a29-20d8174b8d08 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:51.663647 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7549357d-e172-437e-8a29-20d8174b8d08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:51.697656 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-7549357d-e172-437e-8a29-20d8174b8d08 None None] MFA Rules not processed for user `dc5d6e29468d4e0abe5a46f1a28d1722`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:35:51.699317 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-7549357d-e172-437e-8a29-20d8174b8d08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:51.772003 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:35:51.817325 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 973/1923] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:35:51 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:35:52.227528 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c077b6-bf97-4759-a7a5-8432dd0cfda2 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:52.229454 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-d0c077b6-bf97-4759-a7a5-8432dd0cfda2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:52.303135 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c077b6-bf97-4759-a7a5-8432dd0cfda2 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:52.308535 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31a3b10-7313-444b-9f4e-b518191a7c66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:52.309565 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31a3b10-7313-444b-9f4e-b518191a7c66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:52.310004 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31a3b10-7313-444b-9f4e-b518191a7c66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:52.310454 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31a3b10-7313-444b-9f4e-b518191a7c66 admin admin] PATH_INFO: `/v3/users/07acc53c138649e89cdbeabbd3293184` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:52.311899 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31a3b10-7313-444b-9f4e-b518191a7c66 admin admin] RBAC: Authorizing `identity:delete_user(user_id=07acc53c138649e89cdbeabbd3293184)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:52.313798 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31a3b10-7313-444b-9f4e-b518191a7c66 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:52.382553 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=83,keystone:INSERT=4,keystone:DELETE=13 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:35:52.529926 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-a31a3b10-7313-444b-9f4e-b518191a7c66 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07acc53c138649e89cdbeabbd3293184'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:52.655296 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-a31a3b10-7313-444b-9f4e-b518191a7c66 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07acc53c138649e89cdbeabbd3293184'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:52.656047 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-a31a3b10-7313-444b-9f4e-b518191a7c66 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07acc53c138649e89cdbeabbd3293184'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:52.670332 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-a31a3b10-7313-444b-9f4e-b518191a7c66 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07acc53c138649e89cdbeabbd3293184'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:52.671303 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-a31a3b10-7313-444b-9f4e-b518191a7c66 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07acc53c138649e89cdbeabbd3293184'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:52.685383 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 951/1924] 10.222.0.66 () {58 vars in 1223 bytes} [Mon Jul 18 02:35:52 2022] DELETE /identity/v3/users/07acc53c138649e89cdbeabbd3293184 => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:35:52.691505 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b068d9-ccf5-4ef5-842c-ac910cc2d483 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:52.692003 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b068d9-ccf5-4ef5-842c-ac910cc2d483 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:52.692268 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b068d9-ccf5-4ef5-842c-ac910cc2d483 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:52.714393 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-07b068d9-ccf5-4ef5-842c-ac910cc2d483 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:35:52.714393 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-07b068d9-ccf5-4ef5-842c-ac910cc2d483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:52.799701 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 974/1925] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:35:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:35:52.812609 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31a3b10-7313-444b-9f4e-b518191a7c66 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:52.814159 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-a31a3b10-7313-444b-9f4e-b518191a7c66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:52.889118 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b068d9-ccf5-4ef5-842c-ac910cc2d483 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:52.890647 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-07b068d9-ccf5-4ef5-842c-ac910cc2d483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:52.904338 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31a3b10-7313-444b-9f4e-b518191a7c66 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:52.915081 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0e8379-f42f-4d35-bcb1-2fbd7a26f443 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eaf19a259c2741cea375a38763e33cdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:52.915791 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0e8379-f42f-4d35-bcb1-2fbd7a26f443 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:52.916077 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0e8379-f42f-4d35-bcb1-2fbd7a26f443 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:52.917073 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0e8379-f42f-4d35-bcb1-2fbd7a26f443 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:52.945345 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0e8379-f42f-4d35-bcb1-2fbd7a26f443 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:52.946890 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0e8379-f42f-4d35-bcb1-2fbd7a26f443 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:52.970509 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 952/1926] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:35:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:35:52.993052 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b068d9-ccf5-4ef5-842c-ac910cc2d483 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:52.998424 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051900db-6352-4749-a811-0cdc7094535c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd49388facdf446885e55b45ef91f01e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:52.999206 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051900db-6352-4749-a811-0cdc7094535c service cinder] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:52.999825 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051900db-6352-4749-a811-0cdc7094535c service cinder] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:53.000315 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051900db-6352-4749-a811-0cdc7094535c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:53.002810 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-051900db-6352-4749-a811-0cdc7094535c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:53.078813 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051900db-6352-4749-a811-0cdc7094535c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:53.080517 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051900db-6352-4749-a811-0cdc7094535c service cinder] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:53.093406 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 975/1927] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:35:52 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 02:35:53.543684 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:35:53.563731 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0e8379-f42f-4d35-bcb1-2fbd7a26f443 service neutron] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:53.565873 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-5d0e8379-f42f-4d35-bcb1-2fbd7a26f443 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:53.590764 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0e8379-f42f-4d35-bcb1-2fbd7a26f443 service neutron] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:53.598955 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79492b7f-bf28-4f42-a91f-0b2efe3cda2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:53.600262 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79492b7f-bf28-4f42-a91f-0b2efe3cda2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:53.600634 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79492b7f-bf28-4f42-a91f-0b2efe3cda2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:53.602123 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79492b7f-bf28-4f42-a91f-0b2efe3cda2c admin admin] PATH_INFO: `/v3/projects/fe155f52b80347b6be75b811ba524b08` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:53.604289 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79492b7f-bf28-4f42-a91f-0b2efe3cda2c admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe155f52b80347b6be75b811ba524b08)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:53.607037 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79492b7f-bf28-4f42-a91f-0b2efe3cda2c admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:53.617486 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051900db-6352-4749-a811-0cdc7094535c service cinder] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:53.619040 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-051900db-6352-4749-a811-0cdc7094535c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:53.635758 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051900db-6352-4749-a811-0cdc7094535c service cinder] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:53.646529 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202f5777-c46e-4f69-ba2d-2b4429377e9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:53.647114 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202f5777-c46e-4f69-ba2d-2b4429377e9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:53.647405 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202f5777-c46e-4f69-ba2d-2b4429377e9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:53.647688 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202f5777-c46e-4f69-ba2d-2b4429377e9e admin admin] PATH_INFO: `/v3/users/1c1a3587e5a844228d482ffe977cf573` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:53.651531 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202f5777-c46e-4f69-ba2d-2b4429377e9e admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c1a3587e5a844228d482ffe977cf573)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:53.653888 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202f5777-c46e-4f69-ba2d-2b4429377e9e admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:53.897808 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-202f5777-c46e-4f69-ba2d-2b4429377e9e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c1a3587e5a844228d482ffe977cf573'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:53.907274 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-79492b7f-bf28-4f42-a91f-0b2efe3cda2c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe155f52b80347b6be75b811ba524b08', 'request_id': 'req-79492b7f-bf28-4f42-a91f-0b2efe3cda2c'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:53.917438 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-79492b7f-bf28-4f42-a91f-0b2efe3cda2c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe155f52b80347b6be75b811ba524b08', 'request_id': 'req-79492b7f-bf28-4f42-a91f-0b2efe3cda2c'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:53.927646 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-79492b7f-bf28-4f42-a91f-0b2efe3cda2c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe155f52b80347b6be75b811ba524b08', 'request_id': 'req-79492b7f-bf28-4f42-a91f-0b2efe3cda2c'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:53.983114 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-202f5777-c46e-4f69-ba2d-2b4429377e9e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c1a3587e5a844228d482ffe977cf573'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:53.994113 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-202f5777-c46e-4f69-ba2d-2b4429377e9e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c1a3587e5a844228d482ffe977cf573'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:53.995898 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-202f5777-c46e-4f69-ba2d-2b4429377e9e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c1a3587e5a844228d482ffe977cf573'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:54.013402 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-202f5777-c46e-4f69-ba2d-2b4429377e9e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c1a3587e5a844228d482ffe977cf573'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:54.023748 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 976/1928] 10.222.0.66 () {58 vars in 1223 bytes} [Mon Jul 18 02:35:53 2022] DELETE /identity/v3/users/1c1a3587e5a844228d482ffe977cf573 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 02:35:54.028159 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-79492b7f-bf28-4f42-a91f-0b2efe3cda2c admin admin] The token cache is being invalidate because project fe155f52b80347b6be75b811ba524b08 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 02:35:54.028518 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-79492b7f-bf28-4f42-a91f-0b2efe3cda2c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:54.029459 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63eb9ddd-d752-46ec-a833-8801e746cd29 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:54.030314 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63eb9ddd-d752-46ec-a833-8801e746cd29 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:54.030802 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63eb9ddd-d752-46ec-a833-8801e746cd29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:54.031793 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-79492b7f-bf28-4f42-a91f-0b2efe3cda2c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:54.045160 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 953/1929] 10.222.0.66 () {58 vars in 1232 bytes} [Mon Jul 18 02:35:53 2022] DELETE /identity/v3/projects/fe155f52b80347b6be75b811ba524b08 => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:35:54.046225 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79492b7f-bf28-4f42-a91f-0b2efe3cda2c admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:54.049643 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-79492b7f-bf28-4f42-a91f-0b2efe3cda2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:54.072329 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-63eb9ddd-d752-46ec-a833-8801e746cd29 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:35:54.073740 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-63eb9ddd-d752-46ec-a833-8801e746cd29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:54.129342 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79492b7f-bf28-4f42-a91f-0b2efe3cda2c admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:54.154341 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 977/1930] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:35:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:35:54.166741 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63eb9ddd-d752-46ec-a833-8801e746cd29 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:54.172126 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-63eb9ddd-d752-46ec-a833-8801e746cd29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:54.208450 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dfa86d-4c78-4cb3-8896-9d1870f6df48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:54.209424 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dfa86d-4c78-4cb3-8896-9d1870f6df48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:54.209879 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dfa86d-4c78-4cb3-8896-9d1870f6df48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:54.210167 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dfa86d-4c78-4cb3-8896-9d1870f6df48 admin admin] PATH_INFO: `/v3/users/f374e81e77f24d48aea52effd0164e7c` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:54.211387 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dfa86d-4c78-4cb3-8896-9d1870f6df48 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f374e81e77f24d48aea52effd0164e7c)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:54.213936 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dfa86d-4c78-4cb3-8896-9d1870f6df48 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:54.239801 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63eb9ddd-d752-46ec-a833-8801e746cd29 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:54.243599 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60ee01c-141a-4b23-a0b0-0562c47322d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eaf19a259c2741cea375a38763e33cdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:54.247000 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60ee01c-141a-4b23-a0b0-0562c47322d1 service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:54.247291 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60ee01c-141a-4b23-a0b0-0562c47322d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:54.247570 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60ee01c-141a-4b23-a0b0-0562c47322d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:54.257234 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60ee01c-141a-4b23-a0b0-0562c47322d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:54.258781 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60ee01c-141a-4b23-a0b0-0562c47322d1 service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:54.269166 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 978/1931] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:35:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:35:54.485768 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:35:54.504623 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-e6dfa86d-4c78-4cb3-8896-9d1870f6df48 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f374e81e77f24d48aea52effd0164e7c'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:54.604989 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-e6dfa86d-4c78-4cb3-8896-9d1870f6df48 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f374e81e77f24d48aea52effd0164e7c'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:54.606217 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-e6dfa86d-4c78-4cb3-8896-9d1870f6df48 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f374e81e77f24d48aea52effd0164e7c'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:54.624222 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-e6dfa86d-4c78-4cb3-8896-9d1870f6df48 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f374e81e77f24d48aea52effd0164e7c'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:54.625352 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-e6dfa86d-4c78-4cb3-8896-9d1870f6df48 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f374e81e77f24d48aea52effd0164e7c'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:54.641882 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 954/1932] 10.222.0.66 () {58 vars in 1223 bytes} [Mon Jul 18 02:35:54 2022] DELETE /identity/v3/users/f374e81e77f24d48aea52effd0164e7c => generated 0 bytes in 596 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:35:54.832282 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:35:55.076211 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60ee01c-141a-4b23-a0b0-0562c47322d1 service neutron] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:55.077752 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-b60ee01c-141a-4b23-a0b0-0562c47322d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:55.097417 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60ee01c-141a-4b23-a0b0-0562c47322d1 service neutron] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:55.253173 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81242b88-2901-4ebb-a868-560247cfccec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:55.254309 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81242b88-2901-4ebb-a868-560247cfccec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:55.254594 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81242b88-2901-4ebb-a868-560247cfccec admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:55.254873 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81242b88-2901-4ebb-a868-560247cfccec admin admin] PATH_INFO: `/v3/projects/955171086a7844a2a6abc66fb56692f7` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:55.256087 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81242b88-2901-4ebb-a868-560247cfccec admin admin] RBAC: Authorizing `identity:delete_project(project_id=955171086a7844a2a6abc66fb56692f7)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:55.258526 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81242b88-2901-4ebb-a868-560247cfccec admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:55.273825 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dfa86d-4c78-4cb3-8896-9d1870f6df48 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:55.276011 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-e6dfa86d-4c78-4cb3-8896-9d1870f6df48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:55.309371 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dfa86d-4c78-4cb3-8896-9d1870f6df48 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:55.315991 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbbd4ec-1b47-41c6-b0a8-d5b6b1f4e1bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:55.317419 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbbd4ec-1b47-41c6-b0a8-d5b6b1f4e1bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:55.325697 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbbd4ec-1b47-41c6-b0a8-d5b6b1f4e1bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:55.325697 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbbd4ec-1b47-41c6-b0a8-d5b6b1f4e1bd admin admin] PATH_INFO: `/v3/projects/6b23c3e795b84ab7b749da296a6f4c3f` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:55.325697 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbbd4ec-1b47-41c6-b0a8-d5b6b1f4e1bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b23c3e795b84ab7b749da296a6f4c3f)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:55.325697 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbbd4ec-1b47-41c6-b0a8-d5b6b1f4e1bd admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:55.396242 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-81242b88-2901-4ebb-a868-560247cfccec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '955171086a7844a2a6abc66fb56692f7', 'request_id': 'req-81242b88-2901-4ebb-a868-560247cfccec'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:55.402739 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-81242b88-2901-4ebb-a868-560247cfccec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '955171086a7844a2a6abc66fb56692f7', 'request_id': 'req-81242b88-2901-4ebb-a868-560247cfccec'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:55.432596 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-efbbd4ec-1b47-41c6-b0a8-d5b6b1f4e1bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b23c3e795b84ab7b749da296a6f4c3f', 'request_id': 'req-efbbd4ec-1b47-41c6-b0a8-d5b6b1f4e1bd'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:55.441629 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-efbbd4ec-1b47-41c6-b0a8-d5b6b1f4e1bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b23c3e795b84ab7b749da296a6f4c3f', 'request_id': 'req-efbbd4ec-1b47-41c6-b0a8-d5b6b1f4e1bd'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:55.450460 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-efbbd4ec-1b47-41c6-b0a8-d5b6b1f4e1bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b23c3e795b84ab7b749da296a6f4c3f', 'request_id': 'req-efbbd4ec-1b47-41c6-b0a8-d5b6b1f4e1bd'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:55.505965 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-81242b88-2901-4ebb-a868-560247cfccec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '955171086a7844a2a6abc66fb56692f7', 'request_id': 'req-81242b88-2901-4ebb-a868-560247cfccec'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:55.517922 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-81242b88-2901-4ebb-a868-560247cfccec admin admin] The token cache is being invalidate because project 955171086a7844a2a6abc66fb56692f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 02:35:55.518166 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-81242b88-2901-4ebb-a868-560247cfccec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:55.518823 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-81242b88-2901-4ebb-a868-560247cfccec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:55.520664 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 979/1933] 10.222.0.66 () {58 vars in 1232 bytes} [Mon Jul 18 02:35:55 2022] DELETE /identity/v3/projects/955171086a7844a2a6abc66fb56692f7 => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:35:55.524549 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81242b88-2901-4ebb-a868-560247cfccec admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:55.526028 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-81242b88-2901-4ebb-a868-560247cfccec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:55.592339 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81242b88-2901-4ebb-a868-560247cfccec admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:55.597504 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-efbbd4ec-1b47-41c6-b0a8-d5b6b1f4e1bd admin admin] The token cache is being invalidate because project 6b23c3e795b84ab7b749da296a6f4c3f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 02:35:55.597504 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-efbbd4ec-1b47-41c6-b0a8-d5b6b1f4e1bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:55.600454 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-efbbd4ec-1b47-41c6-b0a8-d5b6b1f4e1bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:55.602075 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 955/1934] 10.222.0.66 () {58 vars in 1232 bytes} [Mon Jul 18 02:35:55 2022] DELETE /identity/v3/projects/6b23c3e795b84ab7b749da296a6f4c3f => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 02:35:55.608104 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117e75b3-3c11-427f-9591-b77b1b26dfb4 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:55.609128 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117e75b3-3c11-427f-9591-b77b1b26dfb4 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:55.610259 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117e75b3-3c11-427f-9591-b77b1b26dfb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:55.633434 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-117e75b3-3c11-427f-9591-b77b1b26dfb4 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:35:55.639261 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-117e75b3-3c11-427f-9591-b77b1b26dfb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:55.656422 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d185a02-afeb-4d38-97d7-6398915ba9a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:55.657411 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d185a02-afeb-4d38-97d7-6398915ba9a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:55.657764 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d185a02-afeb-4d38-97d7-6398915ba9a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:55.662219 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d185a02-afeb-4d38-97d7-6398915ba9a9 admin admin] PATH_INFO: `/v3/users/b94783458b4f4156a75400800b2327de` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:55.662219 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d185a02-afeb-4d38-97d7-6398915ba9a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b94783458b4f4156a75400800b2327de)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:55.662219 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d185a02-afeb-4d38-97d7-6398915ba9a9 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:55.697818 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 956/1935] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:35:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:35:55.705511 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed21bf7-b3b0-446e-9e4b-2c43c97056e4 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:55.705798 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed21bf7-b3b0-446e-9e4b-2c43c97056e4 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:55.706073 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed21bf7-b3b0-446e-9e4b-2c43c97056e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:55.730602 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-bed21bf7-b3b0-446e-9e4b-2c43c97056e4 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:35:55.731804 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-bed21bf7-b3b0-446e-9e4b-2c43c97056e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:55.740092 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 957/1936] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:35:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 02:35:55.744308 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed21bf7-b3b0-446e-9e4b-2c43c97056e4 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:55.753404 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed21bf7-b3b0-446e-9e4b-2c43c97056e4 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:55.758859 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1489d47e-3257-40fa-939c-29ff7d98ec0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:55.760188 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1489d47e-3257-40fa-939c-29ff7d98ec0a admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:55.760188 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1489d47e-3257-40fa-939c-29ff7d98ec0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:55.760588 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1489d47e-3257-40fa-939c-29ff7d98ec0a admin admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:55.761425 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1489d47e-3257-40fa-939c-29ff7d98ec0a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:35:55.762004 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1489d47e-3257-40fa-939c-29ff7d98ec0a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:55.763749 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1489d47e-3257-40fa-939c-29ff7d98ec0a admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:55.779443 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 958/1937] 10.222.0.66 () {58 vars in 1152 bytes} [Mon Jul 18 02:35:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 02:35:55.783732 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1489d47e-3257-40fa-939c-29ff7d98ec0a admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:55.795188 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1489d47e-3257-40fa-939c-29ff7d98ec0a admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:55.800271 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e1c7eb-942b-4575-8b49-6dbc2067478c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:55.811087 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e1c7eb-942b-4575-8b49-6dbc2067478c admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:55.811087 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e1c7eb-942b-4575-8b49-6dbc2067478c admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:55.811087 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e1c7eb-942b-4575-8b49-6dbc2067478c admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:55.811087 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e1c7eb-942b-4575-8b49-6dbc2067478c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:55.811087 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e1c7eb-942b-4575-8b49-6dbc2067478c admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:55.933171 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 959/1938] 10.222.0.66 () {60 vars in 1152 bytes} [Mon Jul 18 02:35:55 2022] POST /identity/v3/projects => generated 391 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 02:35:55.937148 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e1c7eb-942b-4575-8b49-6dbc2067478c admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:55.939934 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-3d185a02-afeb-4d38-97d7-6398915ba9a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b94783458b4f4156a75400800b2327de'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:56.022616 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e1c7eb-942b-4575-8b49-6dbc2067478c admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:56.028569 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c7ca4a-c156-4bc7-acf8-d00dc4771649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:56.029254 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c7ca4a-c156-4bc7-acf8-d00dc4771649 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:56.029544 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c7ca4a-c156-4bc7-acf8-d00dc4771649 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:56.029824 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c7ca4a-c156-4bc7-acf8-d00dc4771649 admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:56.030553 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c7ca4a-c156-4bc7-acf8-d00dc4771649 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:56.032710 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c7ca4a-c156-4bc7-acf8-d00dc4771649 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:56.037930 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-3d185a02-afeb-4d38-97d7-6398915ba9a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b94783458b4f4156a75400800b2327de'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:56.047455 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-3d185a02-afeb-4d38-97d7-6398915ba9a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b94783458b4f4156a75400800b2327de'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:56.048144 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-3d185a02-afeb-4d38-97d7-6398915ba9a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b94783458b4f4156a75400800b2327de'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:56.062576 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-3d185a02-afeb-4d38-97d7-6398915ba9a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b94783458b4f4156a75400800b2327de'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:56.071153 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 980/1939] 10.222.0.66 () {58 vars in 1223 bytes} [Mon Jul 18 02:35:55 2022] DELETE /identity/v3/users/b94783458b4f4156a75400800b2327de => generated 0 bytes in 546 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 02:35:56.147492 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 960/1940] 10.222.0.66 () {60 vars in 1143 bytes} [Mon Jul 18 02:35:55 2022] POST /identity/v3/users => generated 296 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 02:35:56.152609 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d185a02-afeb-4d38-97d7-6398915ba9a9 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:56.154854 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-3d185a02-afeb-4d38-97d7-6398915ba9a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:56.246562 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=87,keystone:INSERT=5,keystone:DELETE=8 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:35:56.281064 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d185a02-afeb-4d38-97d7-6398915ba9a9 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:56.287841 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f2658c-564e-4881-bd4f-45e1c226cfef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:56.289083 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f2658c-564e-4881-bd4f-45e1c226cfef admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:56.290045 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f2658c-564e-4881-bd4f-45e1c226cfef admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:56.290485 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f2658c-564e-4881-bd4f-45e1c226cfef admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:56.292828 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f2658c-564e-4881-bd4f-45e1c226cfef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:56.295147 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f2658c-564e-4881-bd4f-45e1c226cfef admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:56.312069 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 981/1941] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:35:56 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:35:56.315115 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c7ca4a-c156-4bc7-acf8-d00dc4771649 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:56.335252 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c7ca4a-c156-4bc7-acf8-d00dc4771649 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:56.356317 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47587fe2-f444-428e-9046-e8f2cffec041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:56.357367 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47587fe2-f444-428e-9046-e8f2cffec041 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:56.357840 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47587fe2-f444-428e-9046-e8f2cffec041 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:56.358287 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47587fe2-f444-428e-9046-e8f2cffec041 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:56.359186 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47587fe2-f444-428e-9046-e8f2cffec041 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:56.361033 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47587fe2-f444-428e-9046-e8f2cffec041 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:56.389873 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 961/1942] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:35:56 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:35:56.396224 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f2658c-564e-4881-bd4f-45e1c226cfef admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:56.411104 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f2658c-564e-4881-bd4f-45e1c226cfef admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:56.417928 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9732cdb-8cbf-4420-8c62-bf7173b8c8dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:56.419147 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9732cdb-8cbf-4420-8c62-bf7173b8c8dc admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:56.419607 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9732cdb-8cbf-4420-8c62-bf7173b8c8dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:56.419986 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9732cdb-8cbf-4420-8c62-bf7173b8c8dc admin admin] PATH_INFO: `/v3/projects/05585f3aa55640b99c54c1a34663527e/users/974a1860b2144c7598788d0e601cc73a/roles/65feabf3b27944a8b01a86c8ac608457` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:56.442497 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9732cdb-8cbf-4420-8c62-bf7173b8c8dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=05585f3aa55640b99c54c1a34663527e, user_id=974a1860b2144c7598788d0e601cc73a, role_id=65feabf3b27944a8b01a86c8ac608457)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:56.446483 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9732cdb-8cbf-4420-8c62-bf7173b8c8dc admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:56.570735 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 982/1943] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:35:56 2022] PUT /identity/v3/projects/05585f3aa55640b99c54c1a34663527e/users/974a1860b2144c7598788d0e601cc73a/roles/65feabf3b27944a8b01a86c8ac608457 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:35:56.579678 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb00474-4a55-405c-9792-6724944df5a0 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:56.580113 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb00474-4a55-405c-9792-6724944df5a0 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:56.580550 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb00474-4a55-405c-9792-6724944df5a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:56.615409 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-3eb00474-4a55-405c-9792-6724944df5a0 None None] MFA Rules not processed for user `974a1860b2144c7598788d0e601cc73a`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:35:56.617393 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-3eb00474-4a55-405c-9792-6724944df5a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:56.699849 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 962/1944] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:35:56 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 02:35:56.704772 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9732cdb-8cbf-4420-8c62-bf7173b8c8dc admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:56.793857 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9732cdb-8cbf-4420-8c62-bf7173b8c8dc admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:56.800432 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e503da76-9796-421f-993f-1d84b6f4c06f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:56.801221 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e503da76-9796-421f-993f-1d84b6f4c06f admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:56.801508 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e503da76-9796-421f-993f-1d84b6f4c06f admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:56.801788 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e503da76-9796-421f-993f-1d84b6f4c06f admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:56.802993 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e503da76-9796-421f-993f-1d84b6f4c06f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:56.804870 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e503da76-9796-421f-993f-1d84b6f4c06f admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:57.384471 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 983/1945] 10.222.0.66 () {60 vars in 1152 bytes} [Mon Jul 18 02:35:56 2022] POST /identity/v3/projects => generated 393 bytes in 677 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:35:57.385980 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb00474-4a55-405c-9792-6724944df5a0 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:57.397418 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb00474-4a55-405c-9792-6724944df5a0 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:57.406180 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:35:57.415133 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e503da76-9796-421f-993f-1d84b6f4c06f admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:57.425054 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-e503da76-9796-421f-993f-1d84b6f4c06f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:57.475994 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdda15d4-624c-4e06-b2a9-ff3e78cd4e12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:57.476818 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdda15d4-624c-4e06-b2a9-ff3e78cd4e12 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:57.477116 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdda15d4-624c-4e06-b2a9-ff3e78cd4e12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:57.477395 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdda15d4-624c-4e06-b2a9-ff3e78cd4e12 admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:57.478181 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdda15d4-624c-4e06-b2a9-ff3e78cd4e12 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:57.481972 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdda15d4-624c-4e06-b2a9-ff3e78cd4e12 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:57.504247 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e503da76-9796-421f-993f-1d84b6f4c06f admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:57.509036 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66c6d22-0f42-4c3d-906b-9165813fee2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:57.509948 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66c6d22-0f42-4c3d-906b-9165813fee2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:57.510362 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66c6d22-0f42-4c3d-906b-9165813fee2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:57.510641 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66c6d22-0f42-4c3d-906b-9165813fee2d admin admin] PATH_INFO: `/v3/projects/cb397225f2a3464f917242ab02841844` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:57.511814 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66c6d22-0f42-4c3d-906b-9165813fee2d admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb397225f2a3464f917242ab02841844)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:57.513749 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66c6d22-0f42-4c3d-906b-9165813fee2d admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:57.603970 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 963/1946] 10.222.0.66 () {60 vars in 1143 bytes} [Mon Jul 18 02:35:57 2022] POST /identity/v3/users => generated 303 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 02:35:57.613797 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdda15d4-624c-4e06-b2a9-ff3e78cd4e12 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:57.625885 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdda15d4-624c-4e06-b2a9-ff3e78cd4e12 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:57.632396 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9fa465-3f42-4f3a-bd3a-c30dc3836669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:57.633283 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9fa465-3f42-4f3a-bd3a-c30dc3836669 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:57.633777 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9fa465-3f42-4f3a-bd3a-c30dc3836669 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:57.634271 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9fa465-3f42-4f3a-bd3a-c30dc3836669 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:57.639839 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9fa465-3f42-4f3a-bd3a-c30dc3836669 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:57.639839 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9fa465-3f42-4f3a-bd3a-c30dc3836669 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:57.651047 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 964/1947] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:35:57 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:35:57.655292 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9fa465-3f42-4f3a-bd3a-c30dc3836669 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:57.657303 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-f66c6d22-0f42-4c3d-906b-9165813fee2d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb397225f2a3464f917242ab02841844', 'request_id': 'req-f66c6d22-0f42-4c3d-906b-9165813fee2d'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:57.664169 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-f66c6d22-0f42-4c3d-906b-9165813fee2d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb397225f2a3464f917242ab02841844', 'request_id': 'req-f66c6d22-0f42-4c3d-906b-9165813fee2d'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:57.673579 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9fa465-3f42-4f3a-bd3a-c30dc3836669 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:57.726041 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9513c4-9c58-463c-b351-602d282b50fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:57.727026 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9513c4-9c58-463c-b351-602d282b50fd admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:57.727309 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9513c4-9c58-463c-b351-602d282b50fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:57.727589 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9513c4-9c58-463c-b351-602d282b50fd admin admin] PATH_INFO: `/v3/projects/a91ae5e8f6b448e6ac8f7e3c1bcebd86/users/4e7f82f24e6642f9bf45bf3cdc40c1a4/roles/d4aa3324f0c84887b41d0dc4acf7f30e` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:57.743122 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9513c4-9c58-463c-b351-602d282b50fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=a91ae5e8f6b448e6ac8f7e3c1bcebd86, user_id=4e7f82f24e6642f9bf45bf3cdc40c1a4, role_id=d4aa3324f0c84887b41d0dc4acf7f30e)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:57.747384 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9513c4-9c58-463c-b351-602d282b50fd admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:57.792532 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-f66c6d22-0f42-4c3d-906b-9165813fee2d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb397225f2a3464f917242ab02841844', 'request_id': 'req-f66c6d22-0f42-4c3d-906b-9165813fee2d'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:57.805756 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-f66c6d22-0f42-4c3d-906b-9165813fee2d admin admin] The token cache is being invalidate because project cb397225f2a3464f917242ab02841844 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 02:35:57.806031 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-f66c6d22-0f42-4c3d-906b-9165813fee2d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:57.807567 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-f66c6d22-0f42-4c3d-906b-9165813fee2d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:57.809358 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 984/1948] 10.222.0.66 () {58 vars in 1232 bytes} [Mon Jul 18 02:35:57 2022] DELETE /identity/v3/projects/cb397225f2a3464f917242ab02841844 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:35:57.814464 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4775696d-1933-4279-a171-2433cf7391fb None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:57.819592 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4775696d-1933-4279-a171-2433cf7391fb None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:57.819592 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4775696d-1933-4279-a171-2433cf7391fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:57.841801 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-4775696d-1933-4279-a171-2433cf7391fb None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:35:57.843447 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-4775696d-1933-4279-a171-2433cf7391fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:57.852994 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 985/1949] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:35:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:35:57.862214 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45cb671-8f67-4519-93a1-125410535396 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:57.863089 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45cb671-8f67-4519-93a1-125410535396 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:57.863482 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45cb671-8f67-4519-93a1-125410535396 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:57.875528 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 965/1950] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:35:57 2022] PUT /identity/v3/projects/a91ae5e8f6b448e6ac8f7e3c1bcebd86/users/4e7f82f24e6642f9bf45bf3cdc40c1a4/roles/d4aa3324f0c84887b41d0dc4acf7f30e => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 02:35:57.879763 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9513c4-9c58-463c-b351-602d282b50fd admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:57.881737 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-0d9513c4-9c58-463c-b351-602d282b50fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:57.891878 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-e45cb671-8f67-4519-93a1-125410535396 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:35:57.893601 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-e45cb671-8f67-4519-93a1-125410535396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:57.959875 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9513c4-9c58-463c-b351-602d282b50fd admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:57.965483 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1d2f57-b7a0-4d0a-be4a-644517b3c7ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:57.966197 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1d2f57-b7a0-4d0a-be4a-644517b3c7ee admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:57.966928 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1d2f57-b7a0-4d0a-be4a-644517b3c7ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:57.966928 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1d2f57-b7a0-4d0a-be4a-644517b3c7ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:57.967649 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1d2f57-b7a0-4d0a-be4a-644517b3c7ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:57.969274 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1d2f57-b7a0-4d0a-be4a-644517b3c7ee admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:57.982135 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 966/1951] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:35:57 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:35:57.984842 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1d2f57-b7a0-4d0a-be4a-644517b3c7ee admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:57.996198 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 986/1952] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:35:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 02:35:57.997293 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45cb671-8f67-4519-93a1-125410535396 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:57.999033 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1d2f57-b7a0-4d0a-be4a-644517b3c7ee admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:58.006211 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef5514f-8385-4a2d-90f7-b9838395b7fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:58.007129 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef5514f-8385-4a2d-90f7-b9838395b7fe admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:58.010221 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45cb671-8f67-4519-93a1-125410535396 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:58.010547 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef5514f-8385-4a2d-90f7-b9838395b7fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:58.010948 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef5514f-8385-4a2d-90f7-b9838395b7fe admin admin] PATH_INFO: `/v3/projects/a91ae5e8f6b448e6ac8f7e3c1bcebd86/users/4e7f82f24e6642f9bf45bf3cdc40c1a4/roles/d4aa3324f0c84887b41d0dc4acf7f30e` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:58.013433 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef5514f-8385-4a2d-90f7-b9838395b7fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=a91ae5e8f6b448e6ac8f7e3c1bcebd86, user_id=4e7f82f24e6642f9bf45bf3cdc40c1a4, role_id=d4aa3324f0c84887b41d0dc4acf7f30e)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:58.016402 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad322e9-5cab-4ff6-859d-c9498db203ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:58.016650 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef5514f-8385-4a2d-90f7-b9838395b7fe admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:58.017484 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad322e9-5cab-4ff6-859d-c9498db203ae admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:58.017902 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad322e9-5cab-4ff6-859d-c9498db203ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:58.018260 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad322e9-5cab-4ff6-859d-c9498db203ae admin admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:58.019086 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad322e9-5cab-4ff6-859d-c9498db203ae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:35:58.019741 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad322e9-5cab-4ff6-859d-c9498db203ae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:58.021474 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad322e9-5cab-4ff6-859d-c9498db203ae admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:58.031261 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 967/1953] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:35:57 2022] PUT /identity/v3/projects/a91ae5e8f6b448e6ac8f7e3c1bcebd86/users/4e7f82f24e6642f9bf45bf3cdc40c1a4/roles/d4aa3324f0c84887b41d0dc4acf7f30e => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 02:35:58.044326 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557bc816-2b6e-4d92-aa18-6833e0f28e51 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:58.044326 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557bc816-2b6e-4d92-aa18-6833e0f28e51 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:58.044326 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557bc816-2b6e-4d92-aa18-6833e0f28e51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:58.050141 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 987/1954] 10.222.0.66 () {58 vars in 1152 bytes} [Mon Jul 18 02:35:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:35:58.052532 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad322e9-5cab-4ff6-859d-c9498db203ae admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:58.066053 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad322e9-5cab-4ff6-859d-c9498db203ae admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:58.083360 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-557bc816-2b6e-4d92-aa18-6833e0f28e51 None None] MFA Rules not processed for user `4e7f82f24e6642f9bf45bf3cdc40c1a4`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:35:58.084657 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-557bc816-2b6e-4d92-aa18-6833e0f28e51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:58.129744 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a17288d-62c1-4e35-b4b4-6cc2189fb501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:58.130576 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a17288d-62c1-4e35-b4b4-6cc2189fb501 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:58.130880 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a17288d-62c1-4e35-b4b4-6cc2189fb501 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:58.131150 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a17288d-62c1-4e35-b4b4-6cc2189fb501 admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:58.131988 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a17288d-62c1-4e35-b4b4-6cc2189fb501 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:58.134047 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a17288d-62c1-4e35-b4b4-6cc2189fb501 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:58.167164 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 968/1955] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:35:58 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:35:58.177866 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557bc816-2b6e-4d92-aa18-6833e0f28e51 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:58.179244 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-557bc816-2b6e-4d92-aa18-6833e0f28e51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:58.241064 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557bc816-2b6e-4d92-aa18-6833e0f28e51 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:58.247685 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5877b51-28e9-4775-9471-e302f7c72d1b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd49388facdf446885e55b45ef91f01e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:58.248435 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5877b51-28e9-4775-9471-e302f7c72d1b service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:58.248730 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5877b51-28e9-4775-9471-e302f7c72d1b service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:58.249008 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5877b51-28e9-4775-9471-e302f7c72d1b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:58.259043 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5877b51-28e9-4775-9471-e302f7c72d1b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:58.262647 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5877b51-28e9-4775-9471-e302f7c72d1b service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:58.273177 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 969/1956] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:35:58 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 02:35:58.296148 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 988/1957] 10.222.0.66 () {60 vars in 1152 bytes} [Mon Jul 18 02:35:58 2022] POST /identity/v3/projects => generated 381 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:35:58.297297 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5877b51-28e9-4775-9471-e302f7c72d1b service cinder] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:58.307741 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5877b51-28e9-4775-9471-e302f7c72d1b service cinder] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:58.344669 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a17288d-62c1-4e35-b4b4-6cc2189fb501 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:58.374834 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7c18c6-ca2a-4462-b91f-ea68631be1d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:58.375606 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7c18c6-ca2a-4462-b91f-ea68631be1d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:58.375966 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7c18c6-ca2a-4462-b91f-ea68631be1d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:58.376319 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7c18c6-ca2a-4462-b91f-ea68631be1d4 admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:58.377176 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7c18c6-ca2a-4462-b91f-ea68631be1d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:58.379277 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7c18c6-ca2a-4462-b91f-ea68631be1d4 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:58.416926 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a17288d-62c1-4e35-b4b4-6cc2189fb501 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:58.421654 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab907442-3a16-47f0-bf6b-8d955294a468 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd49388facdf446885e55b45ef91f01e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:58.422533 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab907442-3a16-47f0-bf6b-8d955294a468 service cinder] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:58.422896 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab907442-3a16-47f0-bf6b-8d955294a468 service cinder] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:58.423248 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab907442-3a16-47f0-bf6b-8d955294a468 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:58.425148 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-ab907442-3a16-47f0-bf6b-8d955294a468 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:58.485721 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 970/1958] 10.222.0.66 () {60 vars in 1143 bytes} [Mon Jul 18 02:35:58 2022] POST /identity/v3/users => generated 291 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:35:58.485721 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7c18c6-ca2a-4462-b91f-ea68631be1d4 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:58.517831 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7c18c6-ca2a-4462-b91f-ea68631be1d4 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:58.528389 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab907442-3a16-47f0-bf6b-8d955294a468 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:58.533668 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab907442-3a16-47f0-bf6b-8d955294a468 service cinder] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:58.534385 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2a485a-1893-4266-9b97-89996ac4cc38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:58.535724 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2a485a-1893-4266-9b97-89996ac4cc38 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:58.539177 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2a485a-1893-4266-9b97-89996ac4cc38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:58.539835 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2a485a-1893-4266-9b97-89996ac4cc38 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:58.544307 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2a485a-1893-4266-9b97-89996ac4cc38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:58.546232 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2a485a-1893-4266-9b97-89996ac4cc38 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:58.547032 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 989/1959] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:35:58 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:35:58.547982 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab907442-3a16-47f0-bf6b-8d955294a468 service cinder] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:58.549750 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-ab907442-3a16-47f0-bf6b-8d955294a468 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:58.565254 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab907442-3a16-47f0-bf6b-8d955294a468 service cinder] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:58.570747 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27f914b-2f9c-4eb0-926d-8e5c56c488d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:58.572364 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27f914b-2f9c-4eb0-926d-8e5c56c488d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:58.572861 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27f914b-2f9c-4eb0-926d-8e5c56c488d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:58.573342 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27f914b-2f9c-4eb0-926d-8e5c56c488d4 admin admin] PATH_INFO: `/v3/users/9370658d43684988b40f263a01ceed34` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:58.574744 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27f914b-2f9c-4eb0-926d-8e5c56c488d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9370658d43684988b40f263a01ceed34)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:58.576792 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27f914b-2f9c-4eb0-926d-8e5c56c488d4 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:58.587349 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 971/1960] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:35:58 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 02:35:58.589016 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2a485a-1893-4266-9b97-89996ac4cc38 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:58.619441 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2a485a-1893-4266-9b97-89996ac4cc38 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:58.628930 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b22fe7e-ea10-455d-a8ef-aa7410d9bd05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:58.631522 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b22fe7e-ea10-455d-a8ef-aa7410d9bd05 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:58.632118 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b22fe7e-ea10-455d-a8ef-aa7410d9bd05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:58.632554 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b22fe7e-ea10-455d-a8ef-aa7410d9bd05 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:58.634242 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b22fe7e-ea10-455d-a8ef-aa7410d9bd05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:58.637374 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b22fe7e-ea10-455d-a8ef-aa7410d9bd05 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:58.653123 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 972/1961] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:35:58 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 02:35:58.654623 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b22fe7e-ea10-455d-a8ef-aa7410d9bd05 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:58.656401 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-8b22fe7e-ea10-455d-a8ef-aa7410d9bd05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:58.671787 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b22fe7e-ea10-455d-a8ef-aa7410d9bd05 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:58.675849 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4203917-b349-4d3e-bf29-6aa1d03d40f2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd49388facdf446885e55b45ef91f01e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:58.676702 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4203917-b349-4d3e-bf29-6aa1d03d40f2 service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:58.677150 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4203917-b349-4d3e-bf29-6aa1d03d40f2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:58.677584 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4203917-b349-4d3e-bf29-6aa1d03d40f2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:58.679638 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-e4203917-b349-4d3e-bf29-6aa1d03d40f2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:58.760363 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4203917-b349-4d3e-bf29-6aa1d03d40f2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:58.762677 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4203917-b349-4d3e-bf29-6aa1d03d40f2 service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:58.774822 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 973/1962] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:35:58 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:35:58.776315 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4203917-b349-4d3e-bf29-6aa1d03d40f2 service cinder] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:58.787426 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4203917-b349-4d3e-bf29-6aa1d03d40f2 service cinder] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:58.796380 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e457558-a59e-4f02-99fc-242a7cc44827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:58.796380 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e457558-a59e-4f02-99fc-242a7cc44827 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:58.796380 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e457558-a59e-4f02-99fc-242a7cc44827 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:58.796380 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e457558-a59e-4f02-99fc-242a7cc44827 admin admin] PATH_INFO: `/v3/projects/9d05bc4b8855449ab6266741b42e6059/users/1edc3f961bfd4a83908088c6a4a14e7f/roles/65feabf3b27944a8b01a86c8ac608457` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:58.824178 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e457558-a59e-4f02-99fc-242a7cc44827 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d05bc4b8855449ab6266741b42e6059, user_id=1edc3f961bfd4a83908088c6a4a14e7f, role_id=65feabf3b27944a8b01a86c8ac608457)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:58.826955 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e457558-a59e-4f02-99fc-242a7cc44827 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:58.878754 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-d27f914b-2f9c-4eb0-926d-8e5c56c488d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9370658d43684988b40f263a01ceed34'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:58.962509 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 974/1963] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:35:58 2022] PUT /identity/v3/projects/9d05bc4b8855449ab6266741b42e6059/users/1edc3f961bfd4a83908088c6a4a14e7f/roles/65feabf3b27944a8b01a86c8ac608457 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:35:58.977387 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b05491-f15c-4c8f-9f8d-41d4188a894b None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:58.977671 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b05491-f15c-4c8f-9f8d-41d4188a894b None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:58.977946 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b05491-f15c-4c8f-9f8d-41d4188a894b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:58.985761 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-d27f914b-2f9c-4eb0-926d-8e5c56c488d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9370658d43684988b40f263a01ceed34'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:58.995373 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-d27f914b-2f9c-4eb0-926d-8e5c56c488d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9370658d43684988b40f263a01ceed34'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:59.000746 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-d27f914b-2f9c-4eb0-926d-8e5c56c488d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9370658d43684988b40f263a01ceed34'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:59.022073 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-57b05491-f15c-4c8f-9f8d-41d4188a894b None None] MFA Rules not processed for user `1edc3f961bfd4a83908088c6a4a14e7f`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:35:59.023521 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-d27f914b-2f9c-4eb0-926d-8e5c56c488d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9370658d43684988b40f263a01ceed34'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:59.031240 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-57b05491-f15c-4c8f-9f8d-41d4188a894b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:59.031649 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 990/1964] 10.222.0.66 () {58 vars in 1223 bytes} [Mon Jul 18 02:35:58 2022] DELETE /identity/v3/users/9370658d43684988b40f263a01ceed34 => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:35:59.035873 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c22102f-b012-42f7-88f8-b580a26c354c None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:59.036462 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c22102f-b012-42f7-88f8-b580a26c354c None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:59.036846 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c22102f-b012-42f7-88f8-b580a26c354c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:59.062706 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-9c22102f-b012-42f7-88f8-b580a26c354c None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:35:59.064309 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-9c22102f-b012-42f7-88f8-b580a26c354c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:59.121012 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 975/1965] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:35:58 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 02:35:59.125829 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b05491-f15c-4c8f-9f8d-41d4188a894b None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:59.127630 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-57b05491-f15c-4c8f-9f8d-41d4188a894b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:59.129394 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:35:59.145672 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 991/1966] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:35:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 02:35:59.155982 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c22102f-b012-42f7-88f8-b580a26c354c None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:59.157848 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-9c22102f-b012-42f7-88f8-b580a26c354c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:59.215455 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b05491-f15c-4c8f-9f8d-41d4188a894b None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:59.219068 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c22102f-b012-42f7-88f8-b580a26c354c None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:59.219843 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16acd232-1e03-4013-af20-cfaa4b58cfa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:59.220548 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16acd232-1e03-4013-af20-cfaa4b58cfa8 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:59.220835 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16acd232-1e03-4013-af20-cfaa4b58cfa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:59.221115 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16acd232-1e03-4013-af20-cfaa4b58cfa8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:59.221808 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16acd232-1e03-4013-af20-cfaa4b58cfa8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:59.223831 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16acd232-1e03-4013-af20-cfaa4b58cfa8 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:59.223946 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efbcb49-18b7-4d42-8386-7cd8783ba21f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eaf19a259c2741cea375a38763e33cdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:59.224819 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efbcb49-18b7-4d42-8386-7cd8783ba21f service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:59.225170 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efbcb49-18b7-4d42-8386-7cd8783ba21f service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:59.225642 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efbcb49-18b7-4d42-8386-7cd8783ba21f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:59.235874 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efbcb49-18b7-4d42-8386-7cd8783ba21f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:59.237778 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efbcb49-18b7-4d42-8386-7cd8783ba21f service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:59.249143 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 992/1967] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:35:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:35:59.361554 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 976/1968] 10.222.0.66 () {60 vars in 1152 bytes} [Mon Jul 18 02:35:59 2022] POST /identity/v3/projects => generated 381 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 02:35:59.365443 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efbcb49-18b7-4d42-8386-7cd8783ba21f service neutron] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:59.381895 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efbcb49-18b7-4d42-8386-7cd8783ba21f service neutron] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:59.446900 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfd11d1-b04f-4721-bcf3-3fb40551d35c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:59.446900 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfd11d1-b04f-4721-bcf3-3fb40551d35c admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:59.446900 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfd11d1-b04f-4721-bcf3-3fb40551d35c admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:59.446900 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfd11d1-b04f-4721-bcf3-3fb40551d35c admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:59.447709 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfd11d1-b04f-4721-bcf3-3fb40551d35c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:59.449111 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfd11d1-b04f-4721-bcf3-3fb40551d35c admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:59.474474 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:35:59.618132 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 993/1969] 10.222.0.66 () {60 vars in 1143 bytes} [Mon Jul 18 02:35:59 2022] POST /identity/v3/users => generated 297 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:35:59.622257 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16acd232-1e03-4013-af20-cfaa4b58cfa8 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:59.632773 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16acd232-1e03-4013-af20-cfaa4b58cfa8 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:59.638266 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebf077f-bd55-4b39-bc5f-e8114170cf68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:59.638928 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebf077f-bd55-4b39-bc5f-e8114170cf68 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:59.639212 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebf077f-bd55-4b39-bc5f-e8114170cf68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:59.639490 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebf077f-bd55-4b39-bc5f-e8114170cf68 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:59.640214 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebf077f-bd55-4b39-bc5f-e8114170cf68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:59.641864 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebf077f-bd55-4b39-bc5f-e8114170cf68 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:59.653088 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 977/1970] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:35:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:35:59.657235 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfd11d1-b04f-4721-bcf3-3fb40551d35c admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:59.667513 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfd11d1-b04f-4721-bcf3-3fb40551d35c admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:59.672220 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb02ff3-987a-4b94-83ac-841952d8f262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:59.673106 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb02ff3-987a-4b94-83ac-841952d8f262 admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:59.673394 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb02ff3-987a-4b94-83ac-841952d8f262 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:59.673675 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb02ff3-987a-4b94-83ac-841952d8f262 admin admin] PATH_INFO: `/v3/projects/791f94defb6f4e2d8d151667700e214c/users/733829a4df704cbb90d1af9c7cf8d2f7/roles/d4aa3324f0c84887b41d0dc4acf7f30e` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:59.688868 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb02ff3-987a-4b94-83ac-841952d8f262 admin admin] RBAC: Authorizing `identity:create_grant(project_id=791f94defb6f4e2d8d151667700e214c, user_id=733829a4df704cbb90d1af9c7cf8d2f7, role_id=d4aa3324f0c84887b41d0dc4acf7f30e)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:59.691816 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb02ff3-987a-4b94-83ac-841952d8f262 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:59.701709 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebf077f-bd55-4b39-bc5f-e8114170cf68 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:59.701709 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-aebf077f-bd55-4b39-bc5f-e8114170cf68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:59.715123 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebf077f-bd55-4b39-bc5f-e8114170cf68 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:59.720112 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cc65d7-7515-4a8f-bb9a-8aa2a71648d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:59.730306 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cc65d7-7515-4a8f-bb9a-8aa2a71648d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:59.730306 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cc65d7-7515-4a8f-bb9a-8aa2a71648d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:59.730306 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cc65d7-7515-4a8f-bb9a-8aa2a71648d5 admin admin] PATH_INFO: `/v3/projects/887cb2ba061f404297533137b4c02a6e` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:59.730306 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cc65d7-7515-4a8f-bb9a-8aa2a71648d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=887cb2ba061f404297533137b4c02a6e)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:59.730306 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cc65d7-7515-4a8f-bb9a-8aa2a71648d5 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:59.851645 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 994/1971] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:35:59 2022] PUT /identity/v3/projects/791f94defb6f4e2d8d151667700e214c/users/733829a4df704cbb90d1af9c7cf8d2f7/roles/d4aa3324f0c84887b41d0dc4acf7f30e => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:35:59.856574 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb02ff3-987a-4b94-83ac-841952d8f262 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:59.867146 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-26cc65d7-7515-4a8f-bb9a-8aa2a71648d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '887cb2ba061f404297533137b4c02a6e', 'request_id': 'req-26cc65d7-7515-4a8f-bb9a-8aa2a71648d5'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:59.873587 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-26cc65d7-7515-4a8f-bb9a-8aa2a71648d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '887cb2ba061f404297533137b4c02a6e', 'request_id': 'req-26cc65d7-7515-4a8f-bb9a-8aa2a71648d5'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:59.881266 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-26cc65d7-7515-4a8f-bb9a-8aa2a71648d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '887cb2ba061f404297533137b4c02a6e', 'request_id': 'req-26cc65d7-7515-4a8f-bb9a-8aa2a71648d5'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:59.922020 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb02ff3-987a-4b94-83ac-841952d8f262 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:35:59.976073 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-26cc65d7-7515-4a8f-bb9a-8aa2a71648d5 admin admin] The token cache is being invalidate because project 887cb2ba061f404297533137b4c02a6e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 02:35:59.976450 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-26cc65d7-7515-4a8f-bb9a-8aa2a71648d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:59.977137 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-26cc65d7-7515-4a8f-bb9a-8aa2a71648d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:35:59.980856 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 978/1972] 10.222.0.66 () {58 vars in 1232 bytes} [Mon Jul 18 02:35:59 2022] DELETE /identity/v3/projects/887cb2ba061f404297533137b4c02a6e => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 02:35:59.980967 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3609d03-8ebd-4fb9-842d-ec8ade7cccd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:35:59.980967 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3609d03-8ebd-4fb9-842d-ec8ade7cccd6 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:35:59.980967 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3609d03-8ebd-4fb9-842d-ec8ade7cccd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:35:59.980967 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3609d03-8ebd-4fb9-842d-ec8ade7cccd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:35:59.981626 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3609d03-8ebd-4fb9-842d-ec8ade7cccd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:35:59.983383 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3609d03-8ebd-4fb9-842d-ec8ade7cccd6 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:35:59.984301 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cc65d7-7515-4a8f-bb9a-8aa2a71648d5 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:35:59.985876 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-26cc65d7-7515-4a8f-bb9a-8aa2a71648d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:35:59.995433 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 995/1973] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:35:59 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 02:35:59.999343 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3609d03-8ebd-4fb9-842d-ec8ade7cccd6 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:00.001217 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-e3609d03-8ebd-4fb9-842d-ec8ade7cccd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:00.002078 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cc65d7-7515-4a8f-bb9a-8aa2a71648d5 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:00.008238 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee346df1-2eb6-4911-8d35-019490c6403d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:00.008238 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee346df1-2eb6-4911-8d35-019490c6403d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:00.008480 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee346df1-2eb6-4911-8d35-019490c6403d admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:00.008608 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee346df1-2eb6-4911-8d35-019490c6403d admin admin] PATH_INFO: `/v3/users/3d80fb60b11c4a5a88136042ae549855` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:00.009920 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee346df1-2eb6-4911-8d35-019490c6403d admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d80fb60b11c4a5a88136042ae549855)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:00.011880 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee346df1-2eb6-4911-8d35-019490c6403d admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:00.018465 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3609d03-8ebd-4fb9-842d-ec8ade7cccd6 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:00.023884 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a39bc62-aeb7-40cf-8c26-9ad15e88402b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:00.024949 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a39bc62-aeb7-40cf-8c26-9ad15e88402b admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:00.025345 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a39bc62-aeb7-40cf-8c26-9ad15e88402b admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:00.025733 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a39bc62-aeb7-40cf-8c26-9ad15e88402b admin admin] PATH_INFO: `/v3/projects/791f94defb6f4e2d8d151667700e214c/users/733829a4df704cbb90d1af9c7cf8d2f7/roles/d4aa3324f0c84887b41d0dc4acf7f30e` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:00.028300 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a39bc62-aeb7-40cf-8c26-9ad15e88402b admin admin] RBAC: Authorizing `identity:create_grant(project_id=791f94defb6f4e2d8d151667700e214c, user_id=733829a4df704cbb90d1af9c7cf8d2f7, role_id=d4aa3324f0c84887b41d0dc4acf7f30e)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:00.032037 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a39bc62-aeb7-40cf-8c26-9ad15e88402b admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:00.045931 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 996/1974] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:35:59 2022] PUT /identity/v3/projects/791f94defb6f4e2d8d151667700e214c/users/733829a4df704cbb90d1af9c7cf8d2f7/roles/d4aa3324f0c84887b41d0dc4acf7f30e => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 02:36:00.046917 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a39bc62-aeb7-40cf-8c26-9ad15e88402b admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:00.048463 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-4a39bc62-aeb7-40cf-8c26-9ad15e88402b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:00.113562 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a39bc62-aeb7-40cf-8c26-9ad15e88402b admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:00.118269 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4cb28f-fe41-4ddd-be16-12a034e03d67 tempest-VolumeQuotaClassesTest-1470238400 tempest-VolumeQuotaClassesTest-1470238400-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc5d6e29468d4e0abe5a46f1a28d1722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eb5945573434e27b9b90d5b6099fae9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:00.119660 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4cb28f-fe41-4ddd-be16-12a034e03d67 tempest-VolumeQuotaClassesTest-1470238400 tempest-VolumeQuotaClassesTest-1470238400-project-admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:00.120022 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4cb28f-fe41-4ddd-be16-12a034e03d67 tempest-VolumeQuotaClassesTest-1470238400 tempest-VolumeQuotaClassesTest-1470238400-project-admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:00.120265 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4cb28f-fe41-4ddd-be16-12a034e03d67 tempest-VolumeQuotaClassesTest-1470238400 tempest-VolumeQuotaClassesTest-1470238400-project-admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:00.121005 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4cb28f-fe41-4ddd-be16-12a034e03d67 tempest-VolumeQuotaClassesTest-1470238400 tempest-VolumeQuotaClassesTest-1470238400-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:36:00.121318 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4cb28f-fe41-4ddd-be16-12a034e03d67 tempest-VolumeQuotaClassesTest-1470238400 tempest-VolumeQuotaClassesTest-1470238400-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:00.122959 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4cb28f-fe41-4ddd-be16-12a034e03d67 tempest-VolumeQuotaClassesTest-1470238400 tempest-VolumeQuotaClassesTest-1470238400-project-admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:00.134837 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 997/1975] 10.222.0.66 () {58 vars in 1152 bytes} [Mon Jul 18 02:36:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:36:00.136811 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2cc17d-8f6e-488e-b0ec-9946c4da582a None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:00.137102 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2cc17d-8f6e-488e-b0ec-9946c4da582a None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:00.137430 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2cc17d-8f6e-488e-b0ec-9946c4da582a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:00.162736 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-4e2cc17d-8f6e-488e-b0ec-9946c4da582a None None] MFA Rules not processed for user `733829a4df704cbb90d1af9c7cf8d2f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:36:00.163948 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-4e2cc17d-8f6e-488e-b0ec-9946c4da582a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:00.216917 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-ee346df1-2eb6-4911-8d35-019490c6403d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d80fb60b11c4a5a88136042ae549855'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:00.233433 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 998/1976] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:36:00 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:36:00.234347 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2cc17d-8f6e-488e-b0ec-9946c4da582a None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:00.245026 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2cc17d-8f6e-488e-b0ec-9946c4da582a None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:00.250113 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b33b6a-af16-406e-85b3-a7541de83944 tempest-VolumeQuotaClassesTest-1470238400 tempest-VolumeQuotaClassesTest-1470238400-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc5d6e29468d4e0abe5a46f1a28d1722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eb5945573434e27b9b90d5b6099fae9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:00.250792 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b33b6a-af16-406e-85b3-a7541de83944 tempest-VolumeQuotaClassesTest-1470238400 tempest-VolumeQuotaClassesTest-1470238400-project-admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:00.251086 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b33b6a-af16-406e-85b3-a7541de83944 tempest-VolumeQuotaClassesTest-1470238400 tempest-VolumeQuotaClassesTest-1470238400-project-admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:00.251371 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b33b6a-af16-406e-85b3-a7541de83944 tempest-VolumeQuotaClassesTest-1470238400 tempest-VolumeQuotaClassesTest-1470238400-project-admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:00.252299 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b33b6a-af16-406e-85b3-a7541de83944 tempest-VolumeQuotaClassesTest-1470238400 tempest-VolumeQuotaClassesTest-1470238400-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:00.254663 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b33b6a-af16-406e-85b3-a7541de83944 tempest-VolumeQuotaClassesTest-1470238400 tempest-VolumeQuotaClassesTest-1470238400-project-admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:00.288421 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-ee346df1-2eb6-4911-8d35-019490c6403d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d80fb60b11c4a5a88136042ae549855'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:00.289072 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-ee346df1-2eb6-4911-8d35-019490c6403d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d80fb60b11c4a5a88136042ae549855'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:00.302220 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-ee346df1-2eb6-4911-8d35-019490c6403d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d80fb60b11c4a5a88136042ae549855'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:00.302916 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-ee346df1-2eb6-4911-8d35-019490c6403d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d80fb60b11c4a5a88136042ae549855'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:00.315352 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 979/1977] 10.222.0.66 () {58 vars in 1223 bytes} [Mon Jul 18 02:35:59 2022] DELETE /identity/v3/users/3d80fb60b11c4a5a88136042ae549855 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 02:36:00.316152 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee346df1-2eb6-4911-8d35-019490c6403d admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:00.317458 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-ee346df1-2eb6-4911-8d35-019490c6403d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:00.378421 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 999/1978] 10.222.0.66 () {60 vars in 1152 bytes} [Mon Jul 18 02:36:00 2022] POST /identity/v3/projects => generated 353 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:36:00.381957 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b33b6a-af16-406e-85b3-a7541de83944 tempest-VolumeQuotaClassesTest-1470238400 tempest-VolumeQuotaClassesTest-1470238400-project-admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:00.383995 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-f3b33b6a-af16-406e-85b3-a7541de83944 tempest-VolumeQuotaClassesTest-1470238400 tempest-VolumeQuotaClassesTest-1470238400-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:00.384761 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee346df1-2eb6-4911-8d35-019490c6403d admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:00.450492 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0780b31-6cd6-49d0-9621-59437e7565ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd49388facdf446885e55b45ef91f01e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:00.451160 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0780b31-6cd6-49d0-9621-59437e7565ef service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:00.451447 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0780b31-6cd6-49d0-9621-59437e7565ef service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:00.451790 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0780b31-6cd6-49d0-9621-59437e7565ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:00.453773 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-a0780b31-6cd6-49d0-9621-59437e7565ef service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:00.458403 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b33b6a-af16-406e-85b3-a7541de83944 tempest-VolumeQuotaClassesTest-1470238400 tempest-VolumeQuotaClassesTest-1470238400-project-admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:00.463842 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a19c59-0d84-43ef-b82e-2401f4bf0846 tempest-VolumeQuotaClassesTest-1470238400 tempest-VolumeQuotaClassesTest-1470238400-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc5d6e29468d4e0abe5a46f1a28d1722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eb5945573434e27b9b90d5b6099fae9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:00.469373 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a19c59-0d84-43ef-b82e-2401f4bf0846 tempest-VolumeQuotaClassesTest-1470238400 tempest-VolumeQuotaClassesTest-1470238400-project-admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:00.469373 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a19c59-0d84-43ef-b82e-2401f4bf0846 tempest-VolumeQuotaClassesTest-1470238400 tempest-VolumeQuotaClassesTest-1470238400-project-admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:00.469373 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a19c59-0d84-43ef-b82e-2401f4bf0846 tempest-VolumeQuotaClassesTest-1470238400 tempest-VolumeQuotaClassesTest-1470238400-project-admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:00.469373 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a19c59-0d84-43ef-b82e-2401f4bf0846 tempest-VolumeQuotaClassesTest-1470238400 tempest-VolumeQuotaClassesTest-1470238400-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:36:00.469373 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a19c59-0d84-43ef-b82e-2401f4bf0846 tempest-VolumeQuotaClassesTest-1470238400 tempest-VolumeQuotaClassesTest-1470238400-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:00.469760 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a19c59-0d84-43ef-b82e-2401f4bf0846 tempest-VolumeQuotaClassesTest-1470238400 tempest-VolumeQuotaClassesTest-1470238400-project-admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:00.482238 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1000/1979] 10.222.0.66 () {58 vars in 1152 bytes} [Mon Jul 18 02:36:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:36:00.521313 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0780b31-6cd6-49d0-9621-59437e7565ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:00.522834 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0780b31-6cd6-49d0-9621-59437e7565ef service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:00.533695 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 980/1980] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:36:00 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:36:00.731136 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a19c59-0d84-43ef-b82e-2401f4bf0846 tempest-VolumeQuotaClassesTest-1470238400 tempest-VolumeQuotaClassesTest-1470238400-project-admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:00.746005 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a19c59-0d84-43ef-b82e-2401f4bf0846 tempest-VolumeQuotaClassesTest-1470238400 tempest-VolumeQuotaClassesTest-1470238400-project-admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:00.752515 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e567276-342f-4c0e-b5b0-8a661129067f tempest-VolumeQuotaClassesTest-1470238400 tempest-VolumeQuotaClassesTest-1470238400-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc5d6e29468d4e0abe5a46f1a28d1722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0eb5945573434e27b9b90d5b6099fae9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:00.753484 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e567276-342f-4c0e-b5b0-8a661129067f tempest-VolumeQuotaClassesTest-1470238400 tempest-VolumeQuotaClassesTest-1470238400-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:00.753870 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e567276-342f-4c0e-b5b0-8a661129067f tempest-VolumeQuotaClassesTest-1470238400 tempest-VolumeQuotaClassesTest-1470238400-project-admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:00.754258 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e567276-342f-4c0e-b5b0-8a661129067f tempest-VolumeQuotaClassesTest-1470238400 tempest-VolumeQuotaClassesTest-1470238400-project-admin] PATH_INFO: `/v3/projects/d233ccb4e9ab4447aee0256aa497ddeb` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:00.756564 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e567276-342f-4c0e-b5b0-8a661129067f tempest-VolumeQuotaClassesTest-1470238400 tempest-VolumeQuotaClassesTest-1470238400-project-admin] RBAC: Authorizing `identity:delete_project(project_id=d233ccb4e9ab4447aee0256aa497ddeb)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:00.757713 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e567276-342f-4c0e-b5b0-8a661129067f tempest-VolumeQuotaClassesTest-1470238400 tempest-VolumeQuotaClassesTest-1470238400-project-admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:00.791152 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0780b31-6cd6-49d0-9621-59437e7565ef service cinder] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:00.792746 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-a0780b31-6cd6-49d0-9621-59437e7565ef service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:00.897867 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0780b31-6cd6-49d0-9621-59437e7565ef service cinder] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:00.909125 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-6e567276-342f-4c0e-b5b0-8a661129067f tempest-VolumeQuotaClassesTest-1470238400 tempest-VolumeQuotaClassesTest-1470238400-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd233ccb4e9ab4447aee0256aa497ddeb', 'request_id': 'req-6e567276-342f-4c0e-b5b0-8a661129067f'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:00.918876 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-6e567276-342f-4c0e-b5b0-8a661129067f tempest-VolumeQuotaClassesTest-1470238400 tempest-VolumeQuotaClassesTest-1470238400-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd233ccb4e9ab4447aee0256aa497ddeb', 'request_id': 'req-6e567276-342f-4c0e-b5b0-8a661129067f'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:00.967978 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99378b51-a420-4259-a66f-fb0d4b96244e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:00.969002 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99378b51-a420-4259-a66f-fb0d4b96244e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:00.969390 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99378b51-a420-4259-a66f-fb0d4b96244e admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:00.969761 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99378b51-a420-4259-a66f-fb0d4b96244e admin admin] PATH_INFO: `/v3/projects/4521fed4fd5346faaa44e4201edbf601` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:00.971252 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99378b51-a420-4259-a66f-fb0d4b96244e admin admin] RBAC: Authorizing `identity:delete_project(project_id=4521fed4fd5346faaa44e4201edbf601)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:00.976039 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99378b51-a420-4259-a66f-fb0d4b96244e admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:01.027824 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-6e567276-342f-4c0e-b5b0-8a661129067f tempest-VolumeQuotaClassesTest-1470238400 tempest-VolumeQuotaClassesTest-1470238400-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd233ccb4e9ab4447aee0256aa497ddeb', 'request_id': 'req-6e567276-342f-4c0e-b5b0-8a661129067f'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:01.041919 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-6e567276-342f-4c0e-b5b0-8a661129067f tempest-VolumeQuotaClassesTest-1470238400 tempest-VolumeQuotaClassesTest-1470238400-project-admin] The token cache is being invalidate because project d233ccb4e9ab4447aee0256aa497ddeb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 02:36:01.042314 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-6e567276-342f-4c0e-b5b0-8a661129067f tempest-VolumeQuotaClassesTest-1470238400 tempest-VolumeQuotaClassesTest-1470238400-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:01.043011 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-6e567276-342f-4c0e-b5b0-8a661129067f tempest-VolumeQuotaClassesTest-1470238400 tempest-VolumeQuotaClassesTest-1470238400-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:01.044730 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1001/1981] 10.222.0.66 () {58 vars in 1232 bytes} [Mon Jul 18 02:36:00 2022] DELETE /identity/v3/projects/d233ccb4e9ab4447aee0256aa497ddeb => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:36:01.118417 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-99378b51-a420-4259-a66f-fb0d4b96244e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4521fed4fd5346faaa44e4201edbf601', 'request_id': 'req-99378b51-a420-4259-a66f-fb0d4b96244e'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:01.124071 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-99378b51-a420-4259-a66f-fb0d4b96244e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4521fed4fd5346faaa44e4201edbf601', 'request_id': 'req-99378b51-a420-4259-a66f-fb0d4b96244e'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:01.130683 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-99378b51-a420-4259-a66f-fb0d4b96244e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4521fed4fd5346faaa44e4201edbf601', 'request_id': 'req-99378b51-a420-4259-a66f-fb0d4b96244e'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:01.238070 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-99378b51-a420-4259-a66f-fb0d4b96244e admin admin] The token cache is being invalidate because project 4521fed4fd5346faaa44e4201edbf601 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 02:36:01.238310 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-99378b51-a420-4259-a66f-fb0d4b96244e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:01.239023 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-99378b51-a420-4259-a66f-fb0d4b96244e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:01.240682 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 981/1982] 10.222.0.66 () {58 vars in 1232 bytes} [Mon Jul 18 02:36:00 2022] DELETE /identity/v3/projects/4521fed4fd5346faaa44e4201edbf601 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:36:01.244934 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e567276-342f-4c0e-b5b0-8a661129067f tempest-VolumeQuotaClassesTest-1470238400 tempest-VolumeQuotaClassesTest-1470238400-project-admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:01.246411 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-6e567276-342f-4c0e-b5b0-8a661129067f tempest-VolumeQuotaClassesTest-1470238400 tempest-VolumeQuotaClassesTest-1470238400-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:01.263832 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e567276-342f-4c0e-b5b0-8a661129067f tempest-VolumeQuotaClassesTest-1470238400 tempest-VolumeQuotaClassesTest-1470238400-project-admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:01.324043 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465e7112-9018-4be0-aeb8-114d2fd087a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:01.324923 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465e7112-9018-4be0-aeb8-114d2fd087a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:01.325203 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465e7112-9018-4be0-aeb8-114d2fd087a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:01.325478 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465e7112-9018-4be0-aeb8-114d2fd087a3 admin admin] PATH_INFO: `/v3/users/dbbb06a55c144b82bd4761edddf4b868` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:01.326707 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465e7112-9018-4be0-aeb8-114d2fd087a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dbbb06a55c144b82bd4761edddf4b868)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:01.328717 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465e7112-9018-4be0-aeb8-114d2fd087a3 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:01.411938 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99378b51-a420-4259-a66f-fb0d4b96244e admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:01.413521 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-99378b51-a420-4259-a66f-fb0d4b96244e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:01.426806 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99378b51-a420-4259-a66f-fb0d4b96244e admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:01.432013 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7046803c-3470-4876-89de-4690b58901d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:01.432948 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7046803c-3470-4876-89de-4690b58901d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:01.433294 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7046803c-3470-4876-89de-4690b58901d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:01.433631 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7046803c-3470-4876-89de-4690b58901d9 admin admin] PATH_INFO: `/v3/users/1edc3f961bfd4a83908088c6a4a14e7f` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:01.434922 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7046803c-3470-4876-89de-4690b58901d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1edc3f961bfd4a83908088c6a4a14e7f)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:01.436918 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7046803c-3470-4876-89de-4690b58901d9 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:01.547925 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:36:01.563391 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-465e7112-9018-4be0-aeb8-114d2fd087a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dbbb06a55c144b82bd4761edddf4b868'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:01.637308 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-465e7112-9018-4be0-aeb8-114d2fd087a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dbbb06a55c144b82bd4761edddf4b868'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:01.645028 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-465e7112-9018-4be0-aeb8-114d2fd087a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dbbb06a55c144b82bd4761edddf4b868'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:01.645746 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-465e7112-9018-4be0-aeb8-114d2fd087a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dbbb06a55c144b82bd4761edddf4b868'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:01.652689 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-7046803c-3470-4876-89de-4690b58901d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1edc3f961bfd4a83908088c6a4a14e7f'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:01.659899 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-465e7112-9018-4be0-aeb8-114d2fd087a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dbbb06a55c144b82bd4761edddf4b868'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:01.665912 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1002/1983] 10.222.0.66 () {58 vars in 1223 bytes} [Mon Jul 18 02:36:01 2022] DELETE /identity/v3/users/dbbb06a55c144b82bd4761edddf4b868 => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:36:01.723729 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465e7112-9018-4be0-aeb8-114d2fd087a3 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:01.725209 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-465e7112-9018-4be0-aeb8-114d2fd087a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:01.736202 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-7046803c-3470-4876-89de-4690b58901d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1edc3f961bfd4a83908088c6a4a14e7f'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:01.736948 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-7046803c-3470-4876-89de-4690b58901d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1edc3f961bfd4a83908088c6a4a14e7f'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:01.751396 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-7046803c-3470-4876-89de-4690b58901d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1edc3f961bfd4a83908088c6a4a14e7f'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:01.752355 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-7046803c-3470-4876-89de-4690b58901d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1edc3f961bfd4a83908088c6a4a14e7f'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:01.767076 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 982/1984] 10.222.0.66 () {58 vars in 1223 bytes} [Mon Jul 18 02:36:01 2022] DELETE /identity/v3/users/1edc3f961bfd4a83908088c6a4a14e7f => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 02:36:01.771671 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b439b4-d753-415e-ae67-800aaa089def None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:01.771986 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b439b4-d753-415e-ae67-800aaa089def None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:01.772294 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b439b4-d753-415e-ae67-800aaa089def None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:01.789701 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-c8b439b4-d753-415e-ae67-800aaa089def None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:36:01.791012 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-c8b439b4-d753-415e-ae67-800aaa089def None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:01.792309 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465e7112-9018-4be0-aeb8-114d2fd087a3 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:01.848073 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd8c0b5-b9ba-4b19-8ad3-5eecf9eb7eec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:01.849296 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd8c0b5-b9ba-4b19-8ad3-5eecf9eb7eec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:01.849420 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 983/1985] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:36:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 02:36:01.849917 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd8c0b5-b9ba-4b19-8ad3-5eecf9eb7eec admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:01.849917 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd8c0b5-b9ba-4b19-8ad3-5eecf9eb7eec admin admin] PATH_INFO: `/v3/users/0ece93dcef974adbaaf83f8264a5a4f8` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:01.850834 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd8c0b5-b9ba-4b19-8ad3-5eecf9eb7eec admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ece93dcef974adbaaf83f8264a5a4f8)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:01.862392 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd8c0b5-b9ba-4b19-8ad3-5eecf9eb7eec admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:01.862495 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b439b4-d753-415e-ae67-800aaa089def None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:01.862495 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-c8b439b4-d753-415e-ae67-800aaa089def None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:01.926388 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b439b4-d753-415e-ae67-800aaa089def None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:01.930016 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13054db-4c9a-4a9a-bd0a-8f7553ac1a4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eaf19a259c2741cea375a38763e33cdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:01.930756 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13054db-4c9a-4a9a-bd0a-8f7553ac1a4e service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:01.933659 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13054db-4c9a-4a9a-bd0a-8f7553ac1a4e service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:01.933659 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13054db-4c9a-4a9a-bd0a-8f7553ac1a4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:01.942826 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13054db-4c9a-4a9a-bd0a-8f7553ac1a4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:01.943659 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13054db-4c9a-4a9a-bd0a-8f7553ac1a4e service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:01.956985 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 984/1986] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:36:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 02:36:02.004819 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:36:02.079845 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-dcd8c0b5-b9ba-4b19-8ad3-5eecf9eb7eec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ece93dcef974adbaaf83f8264a5a4f8'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:02.165026 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-dcd8c0b5-b9ba-4b19-8ad3-5eecf9eb7eec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ece93dcef974adbaaf83f8264a5a4f8'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:02.173395 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13054db-4c9a-4a9a-bd0a-8f7553ac1a4e service neutron] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:02.175944 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-c13054db-4c9a-4a9a-bd0a-8f7553ac1a4e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:02.178546 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-dcd8c0b5-b9ba-4b19-8ad3-5eecf9eb7eec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ece93dcef974adbaaf83f8264a5a4f8'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:02.179501 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-dcd8c0b5-b9ba-4b19-8ad3-5eecf9eb7eec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ece93dcef974adbaaf83f8264a5a4f8'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:02.192070 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13054db-4c9a-4a9a-bd0a-8f7553ac1a4e service neutron] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:02.197684 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02773567-b4d2-4c48-974e-b3511add2fa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:02.198192 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-dcd8c0b5-b9ba-4b19-8ad3-5eecf9eb7eec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ece93dcef974adbaaf83f8264a5a4f8'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:02.198830 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02773567-b4d2-4c48-974e-b3511add2fa4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:02.199156 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02773567-b4d2-4c48-974e-b3511add2fa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:02.199433 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02773567-b4d2-4c48-974e-b3511add2fa4 admin admin] PATH_INFO: `/v3/projects/444b3132a7014b32abe735e5c4944941` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:02.200649 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02773567-b4d2-4c48-974e-b3511add2fa4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=444b3132a7014b32abe735e5c4944941)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:02.202642 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02773567-b4d2-4c48-974e-b3511add2fa4 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:02.206534 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1003/1987] 10.222.0.66 () {58 vars in 1223 bytes} [Mon Jul 18 02:36:01 2022] DELETE /identity/v3/users/0ece93dcef974adbaaf83f8264a5a4f8 => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 02:36:02.211617 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae502dac-fb0c-4b1c-873b-e29a8814bfcb None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:02.212542 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae502dac-fb0c-4b1c-873b-e29a8814bfcb None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:02.213145 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae502dac-fb0c-4b1c-873b-e29a8814bfcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:02.239329 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-ae502dac-fb0c-4b1c-873b-e29a8814bfcb None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:36:02.240691 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-ae502dac-fb0c-4b1c-873b-e29a8814bfcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:02.303043 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1004/1988] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:36:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:36:02.313717 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae502dac-fb0c-4b1c-873b-e29a8814bfcb None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:02.315046 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-ae502dac-fb0c-4b1c-873b-e29a8814bfcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:02.327922 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-02773567-b4d2-4c48-974e-b3511add2fa4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '444b3132a7014b32abe735e5c4944941', 'request_id': 'req-02773567-b4d2-4c48-974e-b3511add2fa4'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:02.333475 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-02773567-b4d2-4c48-974e-b3511add2fa4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '444b3132a7014b32abe735e5c4944941', 'request_id': 'req-02773567-b4d2-4c48-974e-b3511add2fa4'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:02.340179 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-02773567-b4d2-4c48-974e-b3511add2fa4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '444b3132a7014b32abe735e5c4944941', 'request_id': 'req-02773567-b4d2-4c48-974e-b3511add2fa4'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:02.374138 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae502dac-fb0c-4b1c-873b-e29a8814bfcb None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:02.389918 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-02773567-b4d2-4c48-974e-b3511add2fa4 admin admin] The token cache is being invalidate because project 444b3132a7014b32abe735e5c4944941 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 02:36:02.390422 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-02773567-b4d2-4c48-974e-b3511add2fa4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:02.392542 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-02773567-b4d2-4c48-974e-b3511add2fa4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:02.396397 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 985/1989] 10.222.0.66 () {58 vars in 1232 bytes} [Mon Jul 18 02:36:02 2022] DELETE /identity/v3/projects/444b3132a7014b32abe735e5c4944941 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:36:02.402163 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02773567-b4d2-4c48-974e-b3511add2fa4 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:02.403838 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-02773567-b4d2-4c48-974e-b3511add2fa4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:02.427752 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e03ab13-30ef-4d70-a32f-31520ad51501 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eaf19a259c2741cea375a38763e33cdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:02.428409 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e03ab13-30ef-4d70-a32f-31520ad51501 service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:02.428691 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e03ab13-30ef-4d70-a32f-31520ad51501 service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:02.428966 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e03ab13-30ef-4d70-a32f-31520ad51501 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:02.430822 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-5e03ab13-30ef-4d70-a32f-31520ad51501 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:02.474893 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02773567-b4d2-4c48-974e-b3511add2fa4 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:02.480669 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeca9e5-58a8-4dec-943a-47b0f712776c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:02.481588 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeca9e5-58a8-4dec-943a-47b0f712776c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:02.481870 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeca9e5-58a8-4dec-943a-47b0f712776c admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:02.482177 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeca9e5-58a8-4dec-943a-47b0f712776c admin admin] PATH_INFO: `/v3/projects/9d05bc4b8855449ab6266741b42e6059` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:02.483399 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebeca9e5-58a8-4dec-943a-47b0f712776c admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d05bc4b8855449ab6266741b42e6059)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:02.485356 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebeca9e5-58a8-4dec-943a-47b0f712776c admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:02.499248 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e03ab13-30ef-4d70-a32f-31520ad51501 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:02.500701 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e03ab13-30ef-4d70-a32f-31520ad51501 service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:02.510509 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1005/1990] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:36:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 02:36:02.511831 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bcc034-ac20-4412-94a3-15acada5ea90 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:02.512089 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bcc034-ac20-4412-94a3-15acada5ea90 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:02.512369 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bcc034-ac20-4412-94a3-15acada5ea90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:02.531196 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-18bcc034-ac20-4412-94a3-15acada5ea90 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:36:02.532369 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-18bcc034-ac20-4412-94a3-15acada5ea90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:02.541113 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1006/1991] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:36:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:36:02.549527 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6bf305-4628-4300-b7de-8948b7b44551 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:02.550180 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6bf305-4628-4300-b7de-8948b7b44551 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:02.550180 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6bf305-4628-4300-b7de-8948b7b44551 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:02.568982 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-ce6bf305-4628-4300-b7de-8948b7b44551 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:36:02.570125 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-ce6bf305-4628-4300-b7de-8948b7b44551 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:02.578136 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1007/1992] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:36:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:36:02.582202 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6bf305-4628-4300-b7de-8948b7b44551 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:02.592094 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6bf305-4628-4300-b7de-8948b7b44551 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:02.596785 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d24433f-bc9e-4d19-bd8d-5cff55414ce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:02.598255 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d24433f-bc9e-4d19-bd8d-5cff55414ce2 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:02.598255 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d24433f-bc9e-4d19-bd8d-5cff55414ce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:02.598255 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d24433f-bc9e-4d19-bd8d-5cff55414ce2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:02.603880 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d24433f-bc9e-4d19-bd8d-5cff55414ce2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:36:02.603880 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d24433f-bc9e-4d19-bd8d-5cff55414ce2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:02.603880 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d24433f-bc9e-4d19-bd8d-5cff55414ce2 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:02.620096 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1008/1993] 10.222.0.66 () {58 vars in 1152 bytes} [Mon Jul 18 02:36:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 02:36:02.624459 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d24433f-bc9e-4d19-bd8d-5cff55414ce2 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:02.632690 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-ebeca9e5-58a8-4dec-943a-47b0f712776c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d05bc4b8855449ab6266741b42e6059', 'request_id': 'req-ebeca9e5-58a8-4dec-943a-47b0f712776c'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:02.634567 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d24433f-bc9e-4d19-bd8d-5cff55414ce2 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:02.646602 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-ebeca9e5-58a8-4dec-943a-47b0f712776c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d05bc4b8855449ab6266741b42e6059', 'request_id': 'req-ebeca9e5-58a8-4dec-943a-47b0f712776c'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:02.654799 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-ebeca9e5-58a8-4dec-943a-47b0f712776c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d05bc4b8855449ab6266741b42e6059', 'request_id': 'req-ebeca9e5-58a8-4dec-943a-47b0f712776c'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:02.698381 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdaf509-17f0-4b1b-86b4-e992c88e430f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:02.699031 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdaf509-17f0-4b1b-86b4-e992c88e430f admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:02.699313 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdaf509-17f0-4b1b-86b4-e992c88e430f admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:02.699617 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdaf509-17f0-4b1b-86b4-e992c88e430f admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:02.700298 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdaf509-17f0-4b1b-86b4-e992c88e430f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:02.702213 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdaf509-17f0-4b1b-86b4-e992c88e430f admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:02.739766 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-ebeca9e5-58a8-4dec-943a-47b0f712776c admin admin] The token cache is being invalidate because project 9d05bc4b8855449ab6266741b42e6059 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 02:36:02.740061 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-ebeca9e5-58a8-4dec-943a-47b0f712776c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:02.740803 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-ebeca9e5-58a8-4dec-943a-47b0f712776c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:02.742633 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 986/1994] 10.222.0.66 () {58 vars in 1232 bytes} [Mon Jul 18 02:36:02 2022] DELETE /identity/v3/projects/9d05bc4b8855449ab6266741b42e6059 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:36:02.749534 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeca9e5-58a8-4dec-943a-47b0f712776c admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:02.751419 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-ebeca9e5-58a8-4dec-943a-47b0f712776c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:02.764616 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeca9e5-58a8-4dec-943a-47b0f712776c admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:02.770542 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6233fe69-b2b5-486e-9f62-7959b621a330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:02.771523 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6233fe69-b2b5-486e-9f62-7959b621a330 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:02.771816 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6233fe69-b2b5-486e-9f62-7959b621a330 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:02.772093 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6233fe69-b2b5-486e-9f62-7959b621a330 admin admin] PATH_INFO: `/v3/users/733829a4df704cbb90d1af9c7cf8d2f7` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:02.773310 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6233fe69-b2b5-486e-9f62-7959b621a330 admin admin] RBAC: Authorizing `identity:delete_user(user_id=733829a4df704cbb90d1af9c7cf8d2f7)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:02.775883 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6233fe69-b2b5-486e-9f62-7959b621a330 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:02.833238 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1009/1995] 10.222.0.66 () {60 vars in 1152 bytes} [Mon Jul 18 02:36:02 2022] POST /identity/v3/projects => generated 383 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 02:36:02.837972 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdaf509-17f0-4b1b-86b4-e992c88e430f admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:02.839919 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-bfdaf509-17f0-4b1b-86b4-e992c88e430f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:02.904526 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdaf509-17f0-4b1b-86b4-e992c88e430f admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:02.909102 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70f88f6-36d5-400d-b4b4-6d6ce097746c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:02.909775 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70f88f6-36d5-400d-b4b4-6d6ce097746c admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:02.910057 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70f88f6-36d5-400d-b4b4-6d6ce097746c admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:02.910381 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70f88f6-36d5-400d-b4b4-6d6ce097746c admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:02.911093 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70f88f6-36d5-400d-b4b4-6d6ce097746c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:02.913298 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70f88f6-36d5-400d-b4b4-6d6ce097746c admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:03.024107 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-6233fe69-b2b5-486e-9f62-7959b621a330 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '733829a4df704cbb90d1af9c7cf8d2f7'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:03.043942 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1010/1996] 10.222.0.66 () {60 vars in 1143 bytes} [Mon Jul 18 02:36:02 2022] POST /identity/v3/users => generated 292 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 02:36:03.044931 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70f88f6-36d5-400d-b4b4-6d6ce097746c admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:03.046618 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-f70f88f6-36d5-400d-b4b4-6d6ce097746c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:03.058748 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70f88f6-36d5-400d-b4b4-6d6ce097746c admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:03.063434 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b830375d-7e57-4e78-9746-4b79c6dcaba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:03.064265 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b830375d-7e57-4e78-9746-4b79c6dcaba0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:03.068639 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b830375d-7e57-4e78-9746-4b79c6dcaba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:03.068639 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b830375d-7e57-4e78-9746-4b79c6dcaba0 admin admin] PATH_INFO: `/v3/projects/e893953bc0df4b7ab334aca2fc15a8e1` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:03.068639 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b830375d-7e57-4e78-9746-4b79c6dcaba0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e893953bc0df4b7ab334aca2fc15a8e1)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:03.068639 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b830375d-7e57-4e78-9746-4b79c6dcaba0 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:03.116735 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-6233fe69-b2b5-486e-9f62-7959b621a330 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '733829a4df704cbb90d1af9c7cf8d2f7'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:03.117393 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-6233fe69-b2b5-486e-9f62-7959b621a330 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '733829a4df704cbb90d1af9c7cf8d2f7'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:03.129793 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-6233fe69-b2b5-486e-9f62-7959b621a330 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '733829a4df704cbb90d1af9c7cf8d2f7'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:03.130462 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-6233fe69-b2b5-486e-9f62-7959b621a330 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '733829a4df704cbb90d1af9c7cf8d2f7'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:03.143575 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 987/1997] 10.222.0.66 () {58 vars in 1223 bytes} [Mon Jul 18 02:36:02 2022] DELETE /identity/v3/users/733829a4df704cbb90d1af9c7cf8d2f7 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 02:36:03.144898 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6233fe69-b2b5-486e-9f62-7959b621a330 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:03.146327 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-6233fe69-b2b5-486e-9f62-7959b621a330 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:03.183856 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-b830375d-7e57-4e78-9746-4b79c6dcaba0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e893953bc0df4b7ab334aca2fc15a8e1', 'request_id': 'req-b830375d-7e57-4e78-9746-4b79c6dcaba0'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:03.191385 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-b830375d-7e57-4e78-9746-4b79c6dcaba0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e893953bc0df4b7ab334aca2fc15a8e1', 'request_id': 'req-b830375d-7e57-4e78-9746-4b79c6dcaba0'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:03.241081 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6233fe69-b2b5-486e-9f62-7959b621a330 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:03.276803 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-b830375d-7e57-4e78-9746-4b79c6dcaba0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e893953bc0df4b7ab334aca2fc15a8e1', 'request_id': 'req-b830375d-7e57-4e78-9746-4b79c6dcaba0'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:03.289065 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-b830375d-7e57-4e78-9746-4b79c6dcaba0 admin admin] The token cache is being invalidate because project e893953bc0df4b7ab334aca2fc15a8e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 02:36:03.291854 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-b830375d-7e57-4e78-9746-4b79c6dcaba0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:03.291854 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-b830375d-7e57-4e78-9746-4b79c6dcaba0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:03.291854 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1011/1998] 10.222.0.66 () {58 vars in 1232 bytes} [Mon Jul 18 02:36:03 2022] DELETE /identity/v3/projects/e893953bc0df4b7ab334aca2fc15a8e1 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:36:03.295491 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b830375d-7e57-4e78-9746-4b79c6dcaba0 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:03.296929 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-b830375d-7e57-4e78-9746-4b79c6dcaba0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:03.298492 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834744f6-b0d3-4a82-8b1b-604aa76235e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:03.299172 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834744f6-b0d3-4a82-8b1b-604aa76235e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:03.299462 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834744f6-b0d3-4a82-8b1b-604aa76235e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:03.299857 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834744f6-b0d3-4a82-8b1b-604aa76235e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:03.300594 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834744f6-b0d3-4a82-8b1b-604aa76235e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:03.302252 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834744f6-b0d3-4a82-8b1b-604aa76235e2 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:03.309832 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b830375d-7e57-4e78-9746-4b79c6dcaba0 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:03.313434 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 988/1999] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:36:03 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:36:03.317881 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf8aa6b-fe94-4ea7-b422-2b7808b1c6ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:03.318359 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834744f6-b0d3-4a82-8b1b-604aa76235e2 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:03.318922 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf8aa6b-fe94-4ea7-b422-2b7808b1c6ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:03.319205 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf8aa6b-fe94-4ea7-b422-2b7808b1c6ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:03.319481 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf8aa6b-fe94-4ea7-b422-2b7808b1c6ff admin admin] PATH_INFO: `/v3/users/dc5d6e29468d4e0abe5a46f1a28d1722` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:03.320241 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-834744f6-b0d3-4a82-8b1b-604aa76235e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:03.320689 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf8aa6b-fe94-4ea7-b422-2b7808b1c6ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc5d6e29468d4e0abe5a46f1a28d1722)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:03.324205 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf8aa6b-fe94-4ea7-b422-2b7808b1c6ff admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:03.333789 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834744f6-b0d3-4a82-8b1b-604aa76235e2 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:03.338941 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838be47a-8e34-46ea-977b-02ae29a9fb91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:03.343506 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838be47a-8e34-46ea-977b-02ae29a9fb91 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:03.343506 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838be47a-8e34-46ea-977b-02ae29a9fb91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:03.343506 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838be47a-8e34-46ea-977b-02ae29a9fb91 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:03.343506 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838be47a-8e34-46ea-977b-02ae29a9fb91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:03.343506 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838be47a-8e34-46ea-977b-02ae29a9fb91 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:03.353262 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 989/2000] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:36:03 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:36:03.357378 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838be47a-8e34-46ea-977b-02ae29a9fb91 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:03.367321 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838be47a-8e34-46ea-977b-02ae29a9fb91 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:03.373220 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1281028-ecc1-467a-accb-ce7c40280154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:03.374138 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1281028-ecc1-467a-accb-ce7c40280154 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:03.374430 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1281028-ecc1-467a-accb-ce7c40280154 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:03.374708 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1281028-ecc1-467a-accb-ce7c40280154 admin admin] PATH_INFO: `/v3/projects/a1ba327f33ee42a4bec848dcad35f42a/users/fd8cff897cb24fa98bde409f7bf604bf/roles/65feabf3b27944a8b01a86c8ac608457` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:03.391115 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1281028-ecc1-467a-accb-ce7c40280154 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1ba327f33ee42a4bec848dcad35f42a, user_id=fd8cff897cb24fa98bde409f7bf604bf, role_id=65feabf3b27944a8b01a86c8ac608457)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:03.393845 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1281028-ecc1-467a-accb-ce7c40280154 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:03.507567 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 990/2001] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:36:03 2022] PUT /identity/v3/projects/a1ba327f33ee42a4bec848dcad35f42a/users/fd8cff897cb24fa98bde409f7bf604bf/roles/65feabf3b27944a8b01a86c8ac608457 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 02:36:03.520441 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c97fee-d360-43c0-b415-1a5191594787 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:03.520731 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c97fee-d360-43c0-b415-1a5191594787 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:03.521012 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c97fee-d360-43c0-b415-1a5191594787 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:03.543166 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-3bf8aa6b-fe94-4ea7-b422-2b7808b1c6ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc5d6e29468d4e0abe5a46f1a28d1722'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:03.551403 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-44c97fee-d360-43c0-b415-1a5191594787 None None] MFA Rules not processed for user `fd8cff897cb24fa98bde409f7bf604bf`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:36:03.552734 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-44c97fee-d360-43c0-b415-1a5191594787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:03.616281 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 991/2002] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:36:03 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 02:36:03.620743 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c97fee-d360-43c0-b415-1a5191594787 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:03.644740 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-3bf8aa6b-fe94-4ea7-b422-2b7808b1c6ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc5d6e29468d4e0abe5a46f1a28d1722'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:03.657189 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-3bf8aa6b-fe94-4ea7-b422-2b7808b1c6ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc5d6e29468d4e0abe5a46f1a28d1722'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:03.658504 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-3bf8aa6b-fe94-4ea7-b422-2b7808b1c6ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc5d6e29468d4e0abe5a46f1a28d1722'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:03.677490 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-3bf8aa6b-fe94-4ea7-b422-2b7808b1c6ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc5d6e29468d4e0abe5a46f1a28d1722'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:03.684494 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1012/2003] 10.222.0.66 () {58 vars in 1223 bytes} [Mon Jul 18 02:36:03 2022] DELETE /identity/v3/users/dc5d6e29468d4e0abe5a46f1a28d1722 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:36:03.685474 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf8aa6b-fe94-4ea7-b422-2b7808b1c6ff admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:03.686932 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-3bf8aa6b-fe94-4ea7-b422-2b7808b1c6ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:03.687913 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c97fee-d360-43c0-b415-1a5191594787 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:03.725684 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:36:03.756416 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2b0c17-3912-4bcd-aaed-3d7558c1671f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:03.759512 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2b0c17-3912-4bcd-aaed-3d7558c1671f admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:03.759512 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2b0c17-3912-4bcd-aaed-3d7558c1671f admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:03.759512 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2b0c17-3912-4bcd-aaed-3d7558c1671f admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:03.759512 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2b0c17-3912-4bcd-aaed-3d7558c1671f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:03.761938 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2b0c17-3912-4bcd-aaed-3d7558c1671f admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:03.764365 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf8aa6b-fe94-4ea7-b422-2b7808b1c6ff admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:03.770127 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a9dfad-f922-40cc-982b-4345f73ae528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:03.771005 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a9dfad-f922-40cc-982b-4345f73ae528 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:03.771371 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a9dfad-f922-40cc-982b-4345f73ae528 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:03.771703 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a9dfad-f922-40cc-982b-4345f73ae528 admin admin] PATH_INFO: `/v3/projects/791f94defb6f4e2d8d151667700e214c` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:03.772902 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a9dfad-f922-40cc-982b-4345f73ae528 admin admin] RBAC: Authorizing `identity:delete_project(project_id=791f94defb6f4e2d8d151667700e214c)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:03.776086 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a9dfad-f922-40cc-982b-4345f73ae528 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:03.875827 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 992/2004] 10.222.0.66 () {60 vars in 1152 bytes} [Mon Jul 18 02:36:03 2022] POST /identity/v3/projects => generated 383 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 02:36:03.876700 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2b0c17-3912-4bcd-aaed-3d7558c1671f admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:03.878562 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-9e2b0c17-3912-4bcd-aaed-3d7558c1671f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:03.887520 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-93a9dfad-f922-40cc-982b-4345f73ae528 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '791f94defb6f4e2d8d151667700e214c', 'request_id': 'req-93a9dfad-f922-40cc-982b-4345f73ae528'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:03.894379 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-93a9dfad-f922-40cc-982b-4345f73ae528 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '791f94defb6f4e2d8d151667700e214c', 'request_id': 'req-93a9dfad-f922-40cc-982b-4345f73ae528'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:03.947985 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2b0c17-3912-4bcd-aaed-3d7558c1671f admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:03.952420 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f581dcd-5f95-4958-b964-6148b4a4146e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:03.953296 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f581dcd-5f95-4958-b964-6148b4a4146e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:03.953579 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f581dcd-5f95-4958-b964-6148b4a4146e admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:03.953848 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f581dcd-5f95-4958-b964-6148b4a4146e admin admin] PATH_INFO: `/v3/users/974a1860b2144c7598788d0e601cc73a` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:03.955091 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f581dcd-5f95-4958-b964-6148b4a4146e admin admin] RBAC: Authorizing `identity:delete_user(user_id=974a1860b2144c7598788d0e601cc73a)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:03.957021 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f581dcd-5f95-4958-b964-6148b4a4146e admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:03.970543 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-93a9dfad-f922-40cc-982b-4345f73ae528 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '791f94defb6f4e2d8d151667700e214c', 'request_id': 'req-93a9dfad-f922-40cc-982b-4345f73ae528'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:03.972073 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:36:03.985278 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-93a9dfad-f922-40cc-982b-4345f73ae528 admin admin] The token cache is being invalidate because project 791f94defb6f4e2d8d151667700e214c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 02:36:03.985573 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-93a9dfad-f922-40cc-982b-4345f73ae528 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:03.986471 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-93a9dfad-f922-40cc-982b-4345f73ae528 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:03.988285 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1013/2005] 10.222.0.66 () {58 vars in 1232 bytes} [Mon Jul 18 02:36:03 2022] DELETE /identity/v3/projects/791f94defb6f4e2d8d151667700e214c => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:36:03.989007 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a9dfad-f922-40cc-982b-4345f73ae528 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:03.990105 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-93a9dfad-f922-40cc-982b-4345f73ae528 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:04.057080 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a9dfad-f922-40cc-982b-4345f73ae528 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:04.061682 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ee2789-d895-43af-b70b-4b27fb1cccc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:04.062386 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ee2789-d895-43af-b70b-4b27fb1cccc9 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:04.062668 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ee2789-d895-43af-b70b-4b27fb1cccc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:04.062942 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ee2789-d895-43af-b70b-4b27fb1cccc9 admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:04.063626 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ee2789-d895-43af-b70b-4b27fb1cccc9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:04.065556 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ee2789-d895-43af-b70b-4b27fb1cccc9 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:04.149518 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1014/2006] 10.222.0.66 () {60 vars in 1143 bytes} [Mon Jul 18 02:36:03 2022] POST /identity/v3/users => generated 298 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:36:04.151536 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3080d69-4aa7-4733-874a-212338dedc7f None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:04.151536 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3080d69-4aa7-4733-874a-212338dedc7f None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:04.152876 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3080d69-4aa7-4733-874a-212338dedc7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:04.162564 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-5f581dcd-5f95-4958-b964-6148b4a4146e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '974a1860b2144c7598788d0e601cc73a'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:04.177231 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-b3080d69-4aa7-4733-874a-212338dedc7f None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:36:04.178544 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-b3080d69-4aa7-4733-874a-212338dedc7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:04.188742 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1015/2007] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:36:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:36:04.189517 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3080d69-4aa7-4733-874a-212338dedc7f None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:04.191161 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-b3080d69-4aa7-4733-874a-212338dedc7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:04.203969 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3080d69-4aa7-4733-874a-212338dedc7f None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:04.208781 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45e56b4-6adb-4130-9c52-32caaca0e233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:04.209643 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45e56b4-6adb-4130-9c52-32caaca0e233 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:04.209928 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45e56b4-6adb-4130-9c52-32caaca0e233 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:04.210775 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45e56b4-6adb-4130-9c52-32caaca0e233 admin admin] PATH_INFO: `/v3/projects/0eb5945573434e27b9b90d5b6099fae9` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:04.212050 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45e56b4-6adb-4130-9c52-32caaca0e233 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0eb5945573434e27b9b90d5b6099fae9)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:04.213980 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45e56b4-6adb-4130-9c52-32caaca0e233 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:04.238642 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-5f581dcd-5f95-4958-b964-6148b4a4146e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '974a1860b2144c7598788d0e601cc73a'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:04.239525 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-5f581dcd-5f95-4958-b964-6148b4a4146e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '974a1860b2144c7598788d0e601cc73a'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:04.253280 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-5f581dcd-5f95-4958-b964-6148b4a4146e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '974a1860b2144c7598788d0e601cc73a'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:04.253991 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-5f581dcd-5f95-4958-b964-6148b4a4146e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '974a1860b2144c7598788d0e601cc73a'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:04.266667 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 993/2008] 10.222.0.66 () {58 vars in 1223 bytes} [Mon Jul 18 02:36:03 2022] DELETE /identity/v3/users/974a1860b2144c7598788d0e601cc73a => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:36:04.267455 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f581dcd-5f95-4958-b964-6148b4a4146e admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:04.269168 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-5f581dcd-5f95-4958-b964-6148b4a4146e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:04.326793 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-e45e56b4-6adb-4130-9c52-32caaca0e233 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0eb5945573434e27b9b90d5b6099fae9', 'request_id': 'req-e45e56b4-6adb-4130-9c52-32caaca0e233'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:04.332066 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f581dcd-5f95-4958-b964-6148b4a4146e admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:04.332293 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-e45e56b4-6adb-4130-9c52-32caaca0e233 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0eb5945573434e27b9b90d5b6099fae9', 'request_id': 'req-e45e56b4-6adb-4130-9c52-32caaca0e233'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:04.389773 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61609820-2aa4-4b0a-98e8-3e7b907c50f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:04.390507 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61609820-2aa4-4b0a-98e8-3e7b907c50f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:04.390796 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61609820-2aa4-4b0a-98e8-3e7b907c50f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:04.391099 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61609820-2aa4-4b0a-98e8-3e7b907c50f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:04.391889 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61609820-2aa4-4b0a-98e8-3e7b907c50f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:04.393469 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61609820-2aa4-4b0a-98e8-3e7b907c50f9 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:04.404312 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 994/2009] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:36:04 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:36:04.407611 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6d5d96-bc55-4f7e-945d-d7cbbae13c32 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:04.407611 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6d5d96-bc55-4f7e-945d-d7cbbae13c32 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:04.407611 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6d5d96-bc55-4f7e-945d-d7cbbae13c32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:04.426425 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-fa6d5d96-bc55-4f7e-945d-d7cbbae13c32 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:36:04.427098 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-e45e56b4-6adb-4130-9c52-32caaca0e233 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0eb5945573434e27b9b90d5b6099fae9', 'request_id': 'req-e45e56b4-6adb-4130-9c52-32caaca0e233'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:04.427682 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-fa6d5d96-bc55-4f7e-945d-d7cbbae13c32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:04.436811 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 995/2010] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:36:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:36:04.438049 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0e29af-140f-464b-90a6-4acb7536cc91 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:04.438368 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0e29af-140f-464b-90a6-4acb7536cc91 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:04.438647 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0e29af-140f-464b-90a6-4acb7536cc91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:04.439916 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-e45e56b4-6adb-4130-9c52-32caaca0e233 admin admin] The token cache is being invalidate because project 0eb5945573434e27b9b90d5b6099fae9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 02:36:04.440215 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-e45e56b4-6adb-4130-9c52-32caaca0e233 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:04.441193 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-e45e56b4-6adb-4130-9c52-32caaca0e233 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:04.442947 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1016/2011] 10.222.0.66 () {58 vars in 1232 bytes} [Mon Jul 18 02:36:04 2022] DELETE /identity/v3/projects/0eb5945573434e27b9b90d5b6099fae9 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:36:04.443685 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45e56b4-6adb-4130-9c52-32caaca0e233 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:04.444984 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-e45e56b4-6adb-4130-9c52-32caaca0e233 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:04.456140 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-be0e29af-140f-464b-90a6-4acb7536cc91 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:36:04.457319 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-be0e29af-140f-464b-90a6-4acb7536cc91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:04.457752 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45e56b4-6adb-4130-9c52-32caaca0e233 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:04.462804 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66be7539-c3de-492b-ab3f-9776899aba50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:04.463695 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66be7539-c3de-492b-ab3f-9776899aba50 admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:04.463982 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66be7539-c3de-492b-ab3f-9776899aba50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:04.464262 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66be7539-c3de-492b-ab3f-9776899aba50 admin admin] PATH_INFO: `/v3/projects/5138f424cfe342c99531707b76e57e2c/users/6e081d54d53f4e94a1829360441519ff/roles/d4aa3324f0c84887b41d0dc4acf7f30e` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:04.465294 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 996/2012] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:36:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:36:04.466163 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0e29af-140f-464b-90a6-4acb7536cc91 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:04.468222 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-be0e29af-140f-464b-90a6-4acb7536cc91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:04.481628 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0e29af-140f-464b-90a6-4acb7536cc91 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:04.484168 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66be7539-c3de-492b-ab3f-9776899aba50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5138f424cfe342c99531707b76e57e2c, user_id=6e081d54d53f4e94a1829360441519ff, role_id=d4aa3324f0c84887b41d0dc4acf7f30e)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:04.486688 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1339bf96-9519-4b75-9e53-7b7c3791b601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:04.486933 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66be7539-c3de-492b-ab3f-9776899aba50 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:04.487489 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1339bf96-9519-4b75-9e53-7b7c3791b601 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:04.487779 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1339bf96-9519-4b75-9e53-7b7c3791b601 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:04.488061 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1339bf96-9519-4b75-9e53-7b7c3791b601 admin admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:04.488786 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1339bf96-9519-4b75-9e53-7b7c3791b601 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:36:04.489109 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1339bf96-9519-4b75-9e53-7b7c3791b601 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:04.490751 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1339bf96-9519-4b75-9e53-7b7c3791b601 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:04.503243 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 997/2013] 10.222.0.66 () {58 vars in 1152 bytes} [Mon Jul 18 02:36:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:36:04.504564 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f74bae0-2920-4c6d-9cee-8f8b90af9f0a None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:04.504859 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f74bae0-2920-4c6d-9cee-8f8b90af9f0a None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:04.505139 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f74bae0-2920-4c6d-9cee-8f8b90af9f0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:04.522458 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-1f74bae0-2920-4c6d-9cee-8f8b90af9f0a None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:36:04.523649 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-1f74bae0-2920-4c6d-9cee-8f8b90af9f0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:04.531549 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 998/2014] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:36:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:36:04.532332 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f74bae0-2920-4c6d-9cee-8f8b90af9f0a None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:04.533560 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-1f74bae0-2920-4c6d-9cee-8f8b90af9f0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:04.577906 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1017/2015] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:36:04 2022] PUT /identity/v3/projects/5138f424cfe342c99531707b76e57e2c/users/6e081d54d53f4e94a1829360441519ff/roles/d4aa3324f0c84887b41d0dc4acf7f30e => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:36:04.578815 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66be7539-c3de-492b-ab3f-9776899aba50 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:04.593079 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f74bae0-2920-4c6d-9cee-8f8b90af9f0a None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:04.646774 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66be7539-c3de-492b-ab3f-9776899aba50 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:04.647335 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6399c0cd-870c-4765-81d6-d72e263f8173 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eaf19a259c2741cea375a38763e33cdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:04.647864 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6399c0cd-870c-4765-81d6-d72e263f8173 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:04.648148 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6399c0cd-870c-4765-81d6-d72e263f8173 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:04.648422 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6399c0cd-870c-4765-81d6-d72e263f8173 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:04.653994 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adab3239-2df3-4bd9-900a-49cf78cdd4e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:04.654768 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adab3239-2df3-4bd9-900a-49cf78cdd4e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:04.655095 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adab3239-2df3-4bd9-900a-49cf78cdd4e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:04.655378 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adab3239-2df3-4bd9-900a-49cf78cdd4e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:04.656076 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adab3239-2df3-4bd9-900a-49cf78cdd4e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:04.658050 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adab3239-2df3-4bd9-900a-49cf78cdd4e5 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:04.659540 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6399c0cd-870c-4765-81d6-d72e263f8173 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:04.663022 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6399c0cd-870c-4765-81d6-d72e263f8173 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:04.672667 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 999/2016] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:36:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:36:04.674056 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e72746-2f13-49dc-8b0e-4d4aff462a61 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:04.674393 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e72746-2f13-49dc-8b0e-4d4aff462a61 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:04.674688 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e72746-2f13-49dc-8b0e-4d4aff462a61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:04.693290 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-e0e72746-2f13-49dc-8b0e-4d4aff462a61 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:36:04.694706 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-e0e72746-2f13-49dc-8b0e-4d4aff462a61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:04.703214 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1000/2017] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:36:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:36:04.704124 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e72746-2f13-49dc-8b0e-4d4aff462a61 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:04.715167 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e72746-2f13-49dc-8b0e-4d4aff462a61 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:04.720080 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a461b44-b767-4351-a575-b7d00b180178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:04.720867 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a461b44-b767-4351-a575-b7d00b180178 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:04.721156 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a461b44-b767-4351-a575-b7d00b180178 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:04.721435 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a461b44-b767-4351-a575-b7d00b180178 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:04.722218 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a461b44-b767-4351-a575-b7d00b180178 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:04.731768 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a461b44-b767-4351-a575-b7d00b180178 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:04.740827 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1001/2018] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:36:04 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:36:04.741190 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a461b44-b767-4351-a575-b7d00b180178 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:04.751677 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a461b44-b767-4351-a575-b7d00b180178 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:04.756595 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948e2c74-8ddd-4d32-8848-9cce6f15c8c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:04.757255 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948e2c74-8ddd-4d32-8848-9cce6f15c8c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:04.757539 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948e2c74-8ddd-4d32-8848-9cce6f15c8c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:04.757858 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948e2c74-8ddd-4d32-8848-9cce6f15c8c4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:04.758565 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948e2c74-8ddd-4d32-8848-9cce6f15c8c4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:36:04.758937 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948e2c74-8ddd-4d32-8848-9cce6f15c8c4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:04.760569 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948e2c74-8ddd-4d32-8848-9cce6f15c8c4 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:04.773311 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1002/2019] 10.222.0.66 () {58 vars in 1152 bytes} [Mon Jul 18 02:36:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:36:04.774225 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948e2c74-8ddd-4d32-8848-9cce6f15c8c4 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:04.781982 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1018/2020] 10.222.0.66 () {60 vars in 1152 bytes} [Mon Jul 18 02:36:04 2022] POST /identity/v3/projects => generated 383 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:36:04.783097 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adab3239-2df3-4bd9-900a-49cf78cdd4e5 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:04.787727 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948e2c74-8ddd-4d32-8848-9cce6f15c8c4 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:04.851391 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44953dab-9b88-417c-a445-1d0abec3c30d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:04.852569 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44953dab-9b88-417c-a445-1d0abec3c30d admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:04.852910 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44953dab-9b88-417c-a445-1d0abec3c30d admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:04.853185 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44953dab-9b88-417c-a445-1d0abec3c30d admin admin] PATH_INFO: `/v3/projects/5138f424cfe342c99531707b76e57e2c/users/6e081d54d53f4e94a1829360441519ff/roles/d4aa3324f0c84887b41d0dc4acf7f30e` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:04.854300 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adab3239-2df3-4bd9-900a-49cf78cdd4e5 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:04.855255 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44953dab-9b88-417c-a445-1d0abec3c30d admin admin] RBAC: Authorizing `identity:create_grant(project_id=5138f424cfe342c99531707b76e57e2c, user_id=6e081d54d53f4e94a1829360441519ff, role_id=d4aa3324f0c84887b41d0dc4acf7f30e)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:04.858010 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44953dab-9b88-417c-a445-1d0abec3c30d admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:04.859126 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a11d2e-b611-4a8a-9ecc-0ea6dab36364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:04.859779 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a11d2e-b611-4a8a-9ecc-0ea6dab36364 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:04.860064 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a11d2e-b611-4a8a-9ecc-0ea6dab36364 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:04.860340 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a11d2e-b611-4a8a-9ecc-0ea6dab36364 admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:04.861033 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a11d2e-b611-4a8a-9ecc-0ea6dab36364 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:04.863059 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a11d2e-b611-4a8a-9ecc-0ea6dab36364 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:04.869586 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1003/2021] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:36:04 2022] PUT /identity/v3/projects/5138f424cfe342c99531707b76e57e2c/users/6e081d54d53f4e94a1829360441519ff/roles/d4aa3324f0c84887b41d0dc4acf7f30e => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:36:04.870493 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44953dab-9b88-417c-a445-1d0abec3c30d admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:04.943412 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1019/2022] 10.222.0.66 () {60 vars in 1152 bytes} [Mon Jul 18 02:36:04 2022] POST /identity/v3/projects => generated 385 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:36:04.947380 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7032384-e7a0-4d57-a5fd-127d108d8c5f None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:04.947380 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7032384-e7a0-4d57-a5fd-127d108d8c5f None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:04.949370 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7032384-e7a0-4d57-a5fd-127d108d8c5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:04.950478 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44953dab-9b88-417c-a445-1d0abec3c30d admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:04.979879 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-f7032384-e7a0-4d57-a5fd-127d108d8c5f None None] MFA Rules not processed for user `6e081d54d53f4e94a1829360441519ff`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:36:04.981211 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-f7032384-e7a0-4d57-a5fd-127d108d8c5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:05.011968 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8077ef1e-adc8-4414-aa53-6168571ce782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:05.012710 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8077ef1e-adc8-4414-aa53-6168571ce782 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:05.013003 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8077ef1e-adc8-4414-aa53-6168571ce782 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:05.013289 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8077ef1e-adc8-4414-aa53-6168571ce782 admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:05.014144 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8077ef1e-adc8-4414-aa53-6168571ce782 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:05.018304 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8077ef1e-adc8-4414-aa53-6168571ce782 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:05.057190 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1020/2023] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:36:04 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:36:05.058027 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7032384-e7a0-4d57-a5fd-127d108d8c5f None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:05.068282 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7032384-e7a0-4d57-a5fd-127d108d8c5f None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:05.073934 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b48eb2e-e777-4416-8fdb-57ca06ca7e7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:05.074705 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b48eb2e-e777-4416-8fdb-57ca06ca7e7e admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:05.074995 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b48eb2e-e777-4416-8fdb-57ca06ca7e7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:05.075275 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b48eb2e-e777-4416-8fdb-57ca06ca7e7e admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:05.075979 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b48eb2e-e777-4416-8fdb-57ca06ca7e7e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:05.078044 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b48eb2e-e777-4416-8fdb-57ca06ca7e7e admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:05.102366 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1004/2024] 10.222.0.66 () {60 vars in 1143 bytes} [Mon Jul 18 02:36:04 2022] POST /identity/v3/users => generated 292 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:36:05.104017 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8077ef1e-adc8-4414-aa53-6168571ce782 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:05.106226 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-8077ef1e-adc8-4414-aa53-6168571ce782 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:05.166726 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8077ef1e-adc8-4414-aa53-6168571ce782 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:05.170854 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4dcb2d-e696-43b9-ab48-44f80f4c0a7a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd49388facdf446885e55b45ef91f01e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:05.171531 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4dcb2d-e696-43b9-ab48-44f80f4c0a7a service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:05.171818 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4dcb2d-e696-43b9-ab48-44f80f4c0a7a service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:05.182866 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4dcb2d-e696-43b9-ab48-44f80f4c0a7a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:05.189270 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1021/2025] 10.222.0.66 () {60 vars in 1143 bytes} [Mon Jul 18 02:36:05 2022] POST /identity/v3/users => generated 293 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:36:05.189270 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b48eb2e-e777-4416-8fdb-57ca06ca7e7e admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:05.198747 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4dcb2d-e696-43b9-ab48-44f80f4c0a7a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:05.200307 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4dcb2d-e696-43b9-ab48-44f80f4c0a7a service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:05.202481 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:36:05.215766 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1005/2026] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:36:05 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:36:05.216630 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4dcb2d-e696-43b9-ab48-44f80f4c0a7a service cinder] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:05.218251 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-8f4dcb2d-e696-43b9-ab48-44f80f4c0a7a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:05.236166 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4dcb2d-e696-43b9-ab48-44f80f4c0a7a service cinder] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:05.240113 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b48eb2e-e777-4416-8fdb-57ca06ca7e7e admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:05.242375 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8472f4-b946-43fd-8f5e-15ef2c647cee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:05.244342 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8472f4-b946-43fd-8f5e-15ef2c647cee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:05.244774 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8472f4-b946-43fd-8f5e-15ef2c647cee admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:05.245416 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8472f4-b946-43fd-8f5e-15ef2c647cee admin admin] PATH_INFO: `/v3/projects/05585f3aa55640b99c54c1a34663527e` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:05.247038 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8472f4-b946-43fd-8f5e-15ef2c647cee admin admin] RBAC: Authorizing `identity:delete_project(project_id=05585f3aa55640b99c54c1a34663527e)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:05.249495 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8472f4-b946-43fd-8f5e-15ef2c647cee admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:05.264348 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dd5dd7-7add-4413-86c2-f3d385b3a339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:05.274311 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dd5dd7-7add-4413-86c2-f3d385b3a339 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:05.274311 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dd5dd7-7add-4413-86c2-f3d385b3a339 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:05.274311 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dd5dd7-7add-4413-86c2-f3d385b3a339 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:05.274311 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dd5dd7-7add-4413-86c2-f3d385b3a339 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:05.274311 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dd5dd7-7add-4413-86c2-f3d385b3a339 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:05.281659 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1022/2027] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:36:05 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:36:05.282572 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dd5dd7-7add-4413-86c2-f3d385b3a339 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:05.293164 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dd5dd7-7add-4413-86c2-f3d385b3a339 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:05.301039 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a48d8f-21e8-4bed-bdd4-c89c498541e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:05.301535 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a48d8f-21e8-4bed-bdd4-c89c498541e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:05.301830 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a48d8f-21e8-4bed-bdd4-c89c498541e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:05.302144 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a48d8f-21e8-4bed-bdd4-c89c498541e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:05.302996 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a48d8f-21e8-4bed-bdd4-c89c498541e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:05.304598 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a48d8f-21e8-4bed-bdd4-c89c498541e8 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:05.316934 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1023/2028] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:36:05 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:36:05.317875 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a48d8f-21e8-4bed-bdd4-c89c498541e8 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:05.328685 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a48d8f-21e8-4bed-bdd4-c89c498541e8 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:05.333284 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73de7c4-3289-4328-84f6-28e2f1259c10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:05.333941 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73de7c4-3289-4328-84f6-28e2f1259c10 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:05.334258 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73de7c4-3289-4328-84f6-28e2f1259c10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:05.334532 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73de7c4-3289-4328-84f6-28e2f1259c10 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:05.335240 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73de7c4-3289-4328-84f6-28e2f1259c10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:05.336889 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73de7c4-3289-4328-84f6-28e2f1259c10 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:05.350683 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1024/2029] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:36:05 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:36:05.351829 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73de7c4-3289-4328-84f6-28e2f1259c10 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:05.364018 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73de7c4-3289-4328-84f6-28e2f1259c10 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:05.365032 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-5b8472f4-b946-43fd-8f5e-15ef2c647cee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05585f3aa55640b99c54c1a34663527e', 'request_id': 'req-5b8472f4-b946-43fd-8f5e-15ef2c647cee'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:05.370805 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-5b8472f4-b946-43fd-8f5e-15ef2c647cee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05585f3aa55640b99c54c1a34663527e', 'request_id': 'req-5b8472f4-b946-43fd-8f5e-15ef2c647cee'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:05.378420 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-5b8472f4-b946-43fd-8f5e-15ef2c647cee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05585f3aa55640b99c54c1a34663527e', 'request_id': 'req-5b8472f4-b946-43fd-8f5e-15ef2c647cee'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:05.425613 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153e97d2-2008-4a55-ac48-f7d0cfd23b0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:05.426339 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153e97d2-2008-4a55-ac48-f7d0cfd23b0c admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:05.426626 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153e97d2-2008-4a55-ac48-f7d0cfd23b0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:05.426901 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153e97d2-2008-4a55-ac48-f7d0cfd23b0c admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:05.427622 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153e97d2-2008-4a55-ac48-f7d0cfd23b0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:05.429214 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153e97d2-2008-4a55-ac48-f7d0cfd23b0c admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:05.445121 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1025/2030] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:36:05 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:36:05.445121 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153e97d2-2008-4a55-ac48-f7d0cfd23b0c admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:05.455928 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153e97d2-2008-4a55-ac48-f7d0cfd23b0c admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:05.460746 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff52c77-c973-41dc-a46d-49ac43dfc0dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:05.461645 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff52c77-c973-41dc-a46d-49ac43dfc0dd admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:05.462624 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff52c77-c973-41dc-a46d-49ac43dfc0dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:05.462906 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff52c77-c973-41dc-a46d-49ac43dfc0dd admin admin] PATH_INFO: `/v3/projects/ab7cf9a77e614266a1c0a908babb8515/users/9c93dbc940cd4deea0fcb99d0ca0c542/roles/65feabf3b27944a8b01a86c8ac608457` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:05.473443 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=85,keystone:DELETE=10,keystone:INSERT=5 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:36:05.483111 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff52c77-c973-41dc-a46d-49ac43dfc0dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab7cf9a77e614266a1c0a908babb8515, user_id=9c93dbc940cd4deea0fcb99d0ca0c542, role_id=65feabf3b27944a8b01a86c8ac608457)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:05.484228 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-5b8472f4-b946-43fd-8f5e-15ef2c647cee admin admin] The token cache is being invalidate because project 05585f3aa55640b99c54c1a34663527e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 02:36:05.484593 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-5b8472f4-b946-43fd-8f5e-15ef2c647cee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:05.485341 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-5b8472f4-b946-43fd-8f5e-15ef2c647cee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:05.485943 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff52c77-c973-41dc-a46d-49ac43dfc0dd admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:05.489841 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1006/2031] 10.222.0.66 () {58 vars in 1232 bytes} [Mon Jul 18 02:36:05 2022] DELETE /identity/v3/projects/05585f3aa55640b99c54c1a34663527e => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:36:05.489841 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8472f4-b946-43fd-8f5e-15ef2c647cee admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:05.491709 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-5b8472f4-b946-43fd-8f5e-15ef2c647cee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:05.504688 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8472f4-b946-43fd-8f5e-15ef2c647cee admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:05.510171 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25aec0a1-9991-4d60-ada8-6d028ce338d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:05.511069 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aec0a1-9991-4d60-ada8-6d028ce338d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:05.511350 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aec0a1-9991-4d60-ada8-6d028ce338d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:05.511624 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aec0a1-9991-4d60-ada8-6d028ce338d5 admin admin] PATH_INFO: `/v3/projects/4c99a298a28d49ed9e413d7cc07eb5e2/users/b723356f42444558a746db34c4ce93cb/roles/65feabf3b27944a8b01a86c8ac608457` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:05.527267 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25aec0a1-9991-4d60-ada8-6d028ce338d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c99a298a28d49ed9e413d7cc07eb5e2, user_id=b723356f42444558a746db34c4ce93cb, role_id=65feabf3b27944a8b01a86c8ac608457)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:05.529919 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25aec0a1-9991-4d60-ada8-6d028ce338d5 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:05.645957 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1026/2032] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:36:05 2022] PUT /identity/v3/projects/ab7cf9a77e614266a1c0a908babb8515/users/9c93dbc940cd4deea0fcb99d0ca0c542/roles/65feabf3b27944a8b01a86c8ac608457 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:36:05.647001 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff52c77-c973-41dc-a46d-49ac43dfc0dd admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:05.653299 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1007/2033] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:36:05 2022] PUT /identity/v3/projects/4c99a298a28d49ed9e413d7cc07eb5e2/users/b723356f42444558a746db34c4ce93cb/roles/65feabf3b27944a8b01a86c8ac608457 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:36:05.654442 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25aec0a1-9991-4d60-ada8-6d028ce338d5 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:05.656440 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-cff52c77-c973-41dc-a46d-49ac43dfc0dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:05.657093 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-25aec0a1-9991-4d60-ada8-6d028ce338d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:05.716609 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff52c77-c973-41dc-a46d-49ac43dfc0dd admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:05.724885 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25aec0a1-9991-4d60-ada8-6d028ce338d5 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:05.730399 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c57a4f2-c72f-453d-b44d-cbd58b1a9748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:05.731274 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c57a4f2-c72f-453d-b44d-cbd58b1a9748 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:05.731554 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c57a4f2-c72f-453d-b44d-cbd58b1a9748 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:05.731828 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c57a4f2-c72f-453d-b44d-cbd58b1a9748 admin admin] PATH_INFO: `/v3/users/4e7f82f24e6642f9bf45bf3cdc40c1a4` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:05.733004 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c57a4f2-c72f-453d-b44d-cbd58b1a9748 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e7f82f24e6642f9bf45bf3cdc40c1a4)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:05.735054 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c57a4f2-c72f-453d-b44d-cbd58b1a9748 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:05.767981 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d3049a-b4a4-4e66-94ac-a8cc47185f0b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd49388facdf446885e55b45ef91f01e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:05.769073 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d3049a-b4a4-4e66-94ac-a8cc47185f0b service cinder] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:05.769364 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d3049a-b4a4-4e66-94ac-a8cc47185f0b service cinder] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:05.769646 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d3049a-b4a4-4e66-94ac-a8cc47185f0b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:05.772640 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-08d3049a-b4a4-4e66-94ac-a8cc47185f0b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:05.834377 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d3049a-b4a4-4e66-94ac-a8cc47185f0b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:05.835824 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d3049a-b4a4-4e66-94ac-a8cc47185f0b service cinder] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:05.845379 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1027/2034] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:36:05 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:36:05.846674 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e99b206-bfad-403a-b236-b95ae19a778b None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:05.846995 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e99b206-bfad-403a-b236-b95ae19a778b None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:05.847243 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e99b206-bfad-403a-b236-b95ae19a778b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:05.875942 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-5e99b206-bfad-403a-b236-b95ae19a778b None None] MFA Rules not processed for user `9c93dbc940cd4deea0fcb99d0ca0c542`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:36:05.877150 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-5e99b206-bfad-403a-b236-b95ae19a778b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:05.942603 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1028/2035] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:36:05 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:36:05.944011 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e9df81-541f-47b1-bc36-06bba04ea5fc None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:05.944302 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e9df81-541f-47b1-bc36-06bba04ea5fc None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:05.944581 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e9df81-541f-47b1-bc36-06bba04ea5fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:05.973526 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-63e9df81-541f-47b1-bc36-06bba04ea5fc None None] MFA Rules not processed for user `b723356f42444558a746db34c4ce93cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:36:05.974762 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-63e9df81-541f-47b1-bc36-06bba04ea5fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:05.992680 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-3c57a4f2-c72f-453d-b44d-cbd58b1a9748 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e7f82f24e6642f9bf45bf3cdc40c1a4'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:06.045973 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1029/2036] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:36:05 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:36:06.050411 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e9df81-541f-47b1-bc36-06bba04ea5fc None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:06.050411 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-63e9df81-541f-47b1-bc36-06bba04ea5fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:06.059858 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e9df81-541f-47b1-bc36-06bba04ea5fc None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:06.063437 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8370721-dd45-4e49-9d6f-853680e8cf01 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd49388facdf446885e55b45ef91f01e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:06.064076 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8370721-dd45-4e49-9d6f-853680e8cf01 service cinder] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:06.064355 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8370721-dd45-4e49-9d6f-853680e8cf01 service cinder] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:06.064634 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8370721-dd45-4e49-9d6f-853680e8cf01 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:06.067181 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-3c57a4f2-c72f-453d-b44d-cbd58b1a9748 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e7f82f24e6642f9bf45bf3cdc40c1a4'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:06.068956 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-3c57a4f2-c72f-453d-b44d-cbd58b1a9748 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e7f82f24e6642f9bf45bf3cdc40c1a4'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:06.073828 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8370721-dd45-4e49-9d6f-853680e8cf01 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:06.075349 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8370721-dd45-4e49-9d6f-853680e8cf01 service cinder] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:06.084664 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1030/2037] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:36:06 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:36:06.084868 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-3c57a4f2-c72f-453d-b44d-cbd58b1a9748 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e7f82f24e6642f9bf45bf3cdc40c1a4'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:06.085432 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-3c57a4f2-c72f-453d-b44d-cbd58b1a9748 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e7f82f24e6642f9bf45bf3cdc40c1a4'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:06.099478 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1008/2038] 10.222.0.66 () {58 vars in 1223 bytes} [Mon Jul 18 02:36:05 2022] DELETE /identity/v3/users/4e7f82f24e6642f9bf45bf3cdc40c1a4 => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:36:06.175243 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8370721-dd45-4e49-9d6f-853680e8cf01 service cinder] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:06.185630 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8370721-dd45-4e49-9d6f-853680e8cf01 service cinder] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:06.239819 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745bdb67-037e-4104-9cf9-aef315767a16 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd49388facdf446885e55b45ef91f01e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:06.240387 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745bdb67-037e-4104-9cf9-aef315767a16 service cinder] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:06.240763 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745bdb67-037e-4104-9cf9-aef315767a16 service cinder] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:06.241041 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745bdb67-037e-4104-9cf9-aef315767a16 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:06.242878 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-745bdb67-037e-4104-9cf9-aef315767a16 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:06.301548 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745bdb67-037e-4104-9cf9-aef315767a16 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:06.303016 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745bdb67-037e-4104-9cf9-aef315767a16 service cinder] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:06.311767 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1031/2039] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:36:06 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 02:36:06.332623 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c57a4f2-c72f-453d-b44d-cbd58b1a9748 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:06.333824 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:36:06.334559 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-3c57a4f2-c72f-453d-b44d-cbd58b1a9748 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:06.421443 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c57a4f2-c72f-453d-b44d-cbd58b1a9748 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:06.427248 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270f4350-35f4-4957-894d-4b612baae654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:06.428321 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270f4350-35f4-4957-894d-4b612baae654 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:06.428912 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270f4350-35f4-4957-894d-4b612baae654 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:06.429356 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270f4350-35f4-4957-894d-4b612baae654 admin admin] PATH_INFO: `/v3/users/b723356f42444558a746db34c4ce93cb` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:06.430712 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270f4350-35f4-4957-894d-4b612baae654 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b723356f42444558a746db34c4ce93cb)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:06.432984 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270f4350-35f4-4957-894d-4b612baae654 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:06.587762 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745bdb67-037e-4104-9cf9-aef315767a16 service cinder] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:06.589614 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-745bdb67-037e-4104-9cf9-aef315767a16 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:06.618841 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745bdb67-037e-4104-9cf9-aef315767a16 service cinder] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:06.625714 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f648773b-2bdf-4138-9438-a686665fef5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:06.626949 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f648773b-2bdf-4138-9438-a686665fef5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:06.627426 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f648773b-2bdf-4138-9438-a686665fef5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:06.628008 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f648773b-2bdf-4138-9438-a686665fef5f admin admin] PATH_INFO: `/v3/projects/a91ae5e8f6b448e6ac8f7e3c1bcebd86` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:06.629417 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f648773b-2bdf-4138-9438-a686665fef5f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a91ae5e8f6b448e6ac8f7e3c1bcebd86)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:06.631691 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f648773b-2bdf-4138-9438-a686665fef5f admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:06.884680 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-270f4350-35f4-4957-894d-4b612baae654 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b723356f42444558a746db34c4ce93cb'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:06.887850 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-f648773b-2bdf-4138-9438-a686665fef5f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a91ae5e8f6b448e6ac8f7e3c1bcebd86', 'request_id': 'req-f648773b-2bdf-4138-9438-a686665fef5f'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:06.893590 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-f648773b-2bdf-4138-9438-a686665fef5f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a91ae5e8f6b448e6ac8f7e3c1bcebd86', 'request_id': 'req-f648773b-2bdf-4138-9438-a686665fef5f'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:07.072947 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-270f4350-35f4-4957-894d-4b612baae654 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b723356f42444558a746db34c4ce93cb'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:07.073683 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-270f4350-35f4-4957-894d-4b612baae654 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b723356f42444558a746db34c4ce93cb'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:07.087692 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-270f4350-35f4-4957-894d-4b612baae654 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b723356f42444558a746db34c4ce93cb'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:07.088703 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-270f4350-35f4-4957-894d-4b612baae654 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b723356f42444558a746db34c4ce93cb'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:07.105360 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1009/2040] 10.222.0.66 () {58 vars in 1223 bytes} [Mon Jul 18 02:36:06 2022] DELETE /identity/v3/users/b723356f42444558a746db34c4ce93cb => generated 0 bytes in 774 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:36:07.106455 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270f4350-35f4-4957-894d-4b612baae654 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:07.110248 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-270f4350-35f4-4957-894d-4b612baae654 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:07.171174 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270f4350-35f4-4957-894d-4b612baae654 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:07.175779 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f199b0-2861-4255-8aa6-207c65b35883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:07.176653 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f199b0-2861-4255-8aa6-207c65b35883 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:07.177031 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f199b0-2861-4255-8aa6-207c65b35883 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:07.177208 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f199b0-2861-4255-8aa6-207c65b35883 admin admin] PATH_INFO: `/v3/users/fd8cff897cb24fa98bde409f7bf604bf` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:07.178973 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f199b0-2861-4255-8aa6-207c65b35883 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd8cff897cb24fa98bde409f7bf604bf)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:07.179600 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-f648773b-2bdf-4138-9438-a686665fef5f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a91ae5e8f6b448e6ac8f7e3c1bcebd86', 'request_id': 'req-f648773b-2bdf-4138-9438-a686665fef5f'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:07.183182 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f199b0-2861-4255-8aa6-207c65b35883 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:07.195314 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-f648773b-2bdf-4138-9438-a686665fef5f admin admin] The token cache is being invalidate because project a91ae5e8f6b448e6ac8f7e3c1bcebd86 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 02:36:07.195613 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-f648773b-2bdf-4138-9438-a686665fef5f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:07.196471 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-f648773b-2bdf-4138-9438-a686665fef5f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:07.198441 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1032/2041] 10.222.0.66 () {58 vars in 1232 bytes} [Mon Jul 18 02:36:06 2022] DELETE /identity/v3/projects/a91ae5e8f6b448e6ac8f7e3c1bcebd86 => generated 0 bytes in 611 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 02:36:07.199476 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d73fad5-5a24-4bdf-8df2-cfde600f4b6d None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:07.199759 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d73fad5-5a24-4bdf-8df2-cfde600f4b6d None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:07.200035 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d73fad5-5a24-4bdf-8df2-cfde600f4b6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:07.222862 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-0d73fad5-5a24-4bdf-8df2-cfde600f4b6d None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:36:07.228129 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-0d73fad5-5a24-4bdf-8df2-cfde600f4b6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:07.233452 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1033/2042] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:36:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:36:07.235945 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7d3c2c-9b39-4a63-b34e-bb39af64ad03 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:07.235945 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7d3c2c-9b39-4a63-b34e-bb39af64ad03 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:07.235945 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7d3c2c-9b39-4a63-b34e-bb39af64ad03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:07.253574 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-ba7d3c2c-9b39-4a63-b34e-bb39af64ad03 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:36:07.254822 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-ba7d3c2c-9b39-4a63-b34e-bb39af64ad03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:07.263696 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1034/2043] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:36:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:36:07.264694 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7d3c2c-9b39-4a63-b34e-bb39af64ad03 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:07.265947 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-ba7d3c2c-9b39-4a63-b34e-bb39af64ad03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:07.330129 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7d3c2c-9b39-4a63-b34e-bb39af64ad03 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:07.334560 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8edb38-4832-43de-b580-e8fb7d1c8df3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eaf19a259c2741cea375a38763e33cdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:07.336814 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8edb38-4832-43de-b580-e8fb7d1c8df3 service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:07.336814 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8edb38-4832-43de-b580-e8fb7d1c8df3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:07.336814 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8edb38-4832-43de-b580-e8fb7d1c8df3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:07.345424 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8edb38-4832-43de-b580-e8fb7d1c8df3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:07.346906 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8edb38-4832-43de-b580-e8fb7d1c8df3 service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:07.359281 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1035/2044] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:36:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:36:07.360298 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a737a4-43a6-4367-977b-033f1665d00c None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:07.360569 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a737a4-43a6-4367-977b-033f1665d00c None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:07.360850 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a737a4-43a6-4367-977b-033f1665d00c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:07.380151 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-99a737a4-43a6-4367-977b-033f1665d00c None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:36:07.381753 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-99a737a4-43a6-4367-977b-033f1665d00c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:07.390900 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1036/2045] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:36:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:36:07.396113 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a737a4-43a6-4367-977b-033f1665d00c None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:07.398518 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-d8f199b0-2861-4255-8aa6-207c65b35883 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd8cff897cb24fa98bde409f7bf604bf'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:07.406185 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a737a4-43a6-4367-977b-033f1665d00c None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:07.413125 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd8cfb7-37ec-4cfa-b808-5d7949916601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:07.413816 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd8cfb7-37ec-4cfa-b808-5d7949916601 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:07.414142 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd8cfb7-37ec-4cfa-b808-5d7949916601 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:07.414419 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd8cfb7-37ec-4cfa-b808-5d7949916601 admin admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:07.416258 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd8cfb7-37ec-4cfa-b808-5d7949916601 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:36:07.416515 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd8cfb7-37ec-4cfa-b808-5d7949916601 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:07.418219 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd8cfb7-37ec-4cfa-b808-5d7949916601 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:07.431046 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1037/2046] 10.222.0.66 () {58 vars in 1152 bytes} [Mon Jul 18 02:36:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:36:07.435371 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd8cfb7-37ec-4cfa-b808-5d7949916601 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:07.445312 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd8cfb7-37ec-4cfa-b808-5d7949916601 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:07.451273 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e255b0cf-a7ba-47db-83a7-792e0f8d0c0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:07.451950 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e255b0cf-a7ba-47db-83a7-792e0f8d0c0f admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:07.452236 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e255b0cf-a7ba-47db-83a7-792e0f8d0c0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:07.452517 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e255b0cf-a7ba-47db-83a7-792e0f8d0c0f admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:07.453255 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e255b0cf-a7ba-47db-83a7-792e0f8d0c0f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:07.455246 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e255b0cf-a7ba-47db-83a7-792e0f8d0c0f admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:07.457104 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:36:07.472908 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-d8f199b0-2861-4255-8aa6-207c65b35883 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd8cff897cb24fa98bde409f7bf604bf'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:07.473570 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-d8f199b0-2861-4255-8aa6-207c65b35883 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd8cff897cb24fa98bde409f7bf604bf'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:07.486879 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-d8f199b0-2861-4255-8aa6-207c65b35883 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd8cff897cb24fa98bde409f7bf604bf'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:07.487570 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-d8f199b0-2861-4255-8aa6-207c65b35883 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd8cff897cb24fa98bde409f7bf604bf'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:07.500158 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1010/2047] 10.222.0.66 () {58 vars in 1223 bytes} [Mon Jul 18 02:36:07 2022] DELETE /identity/v3/users/fd8cff897cb24fa98bde409f7bf604bf => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:36:07.504494 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03cff71-e566-4c6e-9eef-6a528d0589e6 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:07.504784 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03cff71-e566-4c6e-9eef-6a528d0589e6 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:07.505066 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03cff71-e566-4c6e-9eef-6a528d0589e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:07.522790 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-c03cff71-e566-4c6e-9eef-6a528d0589e6 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:36:07.523970 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-c03cff71-e566-4c6e-9eef-6a528d0589e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:07.547038 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1038/2048] 10.222.0.66 () {60 vars in 1152 bytes} [Mon Jul 18 02:36:07 2022] POST /identity/v3/projects => generated 385 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 02:36:07.551155 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e255b0cf-a7ba-47db-83a7-792e0f8d0c0f admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:07.552825 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-e255b0cf-a7ba-47db-83a7-792e0f8d0c0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:07.590717 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1011/2049] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:36:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 02:36:07.603065 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03cff71-e566-4c6e-9eef-6a528d0589e6 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:07.605001 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-c03cff71-e566-4c6e-9eef-6a528d0589e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:07.629888 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e255b0cf-a7ba-47db-83a7-792e0f8d0c0f admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:07.637931 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fd9e17-ae38-4e35-bedb-02d56213f4d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:07.638665 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fd9e17-ae38-4e35-bedb-02d56213f4d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:07.642971 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fd9e17-ae38-4e35-bedb-02d56213f4d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:07.642971 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fd9e17-ae38-4e35-bedb-02d56213f4d5 admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:07.642971 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fd9e17-ae38-4e35-bedb-02d56213f4d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:07.642971 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fd9e17-ae38-4e35-bedb-02d56213f4d5 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:07.669498 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03cff71-e566-4c6e-9eef-6a528d0589e6 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:07.720862 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c40f73-e835-4b76-8f56-69927a8d7aae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eaf19a259c2741cea375a38763e33cdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:07.721673 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c40f73-e835-4b76-8f56-69927a8d7aae service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:07.721963 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c40f73-e835-4b76-8f56-69927a8d7aae service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:07.722290 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c40f73-e835-4b76-8f56-69927a8d7aae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:07.732736 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c40f73-e835-4b76-8f56-69927a8d7aae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:07.734296 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c40f73-e835-4b76-8f56-69927a8d7aae service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:07.735868 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1039/2050] 10.222.0.66 () {60 vars in 1143 bytes} [Mon Jul 18 02:36:07 2022] POST /identity/v3/users => generated 293 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 02:36:07.739872 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fd9e17-ae38-4e35-bedb-02d56213f4d5 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:07.747954 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1012/2051] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:36:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 02:36:07.749469 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fd9e17-ae38-4e35-bedb-02d56213f4d5 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:07.754389 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d40438-934e-4f95-81dd-fae4c070ed66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:07.755058 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d40438-934e-4f95-81dd-fae4c070ed66 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:07.757059 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d40438-934e-4f95-81dd-fae4c070ed66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:07.757059 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d40438-934e-4f95-81dd-fae4c070ed66 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:07.757059 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d40438-934e-4f95-81dd-fae4c070ed66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:07.757991 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d40438-934e-4f95-81dd-fae4c070ed66 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:07.769237 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1040/2052] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:36:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 02:36:07.773177 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c40f73-e835-4b76-8f56-69927a8d7aae service neutron] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:07.783521 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c40f73-e835-4b76-8f56-69927a8d7aae service neutron] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:07.788265 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e20ae52-eab1-4dbd-bbaa-ca72a6fcf087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:07.788925 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e20ae52-eab1-4dbd-bbaa-ca72a6fcf087 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:07.789214 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e20ae52-eab1-4dbd-bbaa-ca72a6fcf087 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:07.789490 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e20ae52-eab1-4dbd-bbaa-ca72a6fcf087 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:07.790943 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e20ae52-eab1-4dbd-bbaa-ca72a6fcf087 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:07.792591 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e20ae52-eab1-4dbd-bbaa-ca72a6fcf087 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:07.802566 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1013/2053] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:36:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:36:07.814207 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d40438-934e-4f95-81dd-fae4c070ed66 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:07.824804 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d40438-934e-4f95-81dd-fae4c070ed66 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:07.829488 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dec16af-dc08-43e6-b611-dc8fa6b537fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:07.830393 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dec16af-dc08-43e6-b611-dc8fa6b537fe admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:07.830679 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dec16af-dc08-43e6-b611-dc8fa6b537fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:07.830956 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dec16af-dc08-43e6-b611-dc8fa6b537fe admin admin] PATH_INFO: `/v3/projects/5477a15859d44821b12789b6dbd124ad/users/3c3f3a5269d446cc8a5cc9c54fbd3643/roles/65feabf3b27944a8b01a86c8ac608457` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:07.847817 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dec16af-dc08-43e6-b611-dc8fa6b537fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=5477a15859d44821b12789b6dbd124ad, user_id=3c3f3a5269d446cc8a5cc9c54fbd3643, role_id=65feabf3b27944a8b01a86c8ac608457)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:07.848857 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e20ae52-eab1-4dbd-bbaa-ca72a6fcf087 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:07.850582 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dec16af-dc08-43e6-b611-dc8fa6b537fe admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:07.850767 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-5e20ae52-eab1-4dbd-bbaa-ca72a6fcf087 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:07.866688 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e20ae52-eab1-4dbd-bbaa-ca72a6fcf087 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:07.873553 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47789c8a-0c23-4aad-98a4-a284c94e2427 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:07.876968 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47789c8a-0c23-4aad-98a4-a284c94e2427 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:07.876968 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47789c8a-0c23-4aad-98a4-a284c94e2427 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:07.876968 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47789c8a-0c23-4aad-98a4-a284c94e2427 admin admin] PATH_INFO: `/v3/projects/4c99a298a28d49ed9e413d7cc07eb5e2` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:07.876968 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47789c8a-0c23-4aad-98a4-a284c94e2427 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c99a298a28d49ed9e413d7cc07eb5e2)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:07.878293 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47789c8a-0c23-4aad-98a4-a284c94e2427 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:07.936382 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1041/2054] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:36:07 2022] PUT /identity/v3/projects/5477a15859d44821b12789b6dbd124ad/users/3c3f3a5269d446cc8a5cc9c54fbd3643/roles/65feabf3b27944a8b01a86c8ac608457 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:36:07.947925 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8980b1f4-4856-47da-b909-4eab45492c12 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:07.948208 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8980b1f4-4856-47da-b909-4eab45492c12 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:07.948479 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8980b1f4-4856-47da-b909-4eab45492c12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:07.974026 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-8980b1f4-4856-47da-b909-4eab45492c12 None None] MFA Rules not processed for user `3c3f3a5269d446cc8a5cc9c54fbd3643`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:36:07.975252 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-8980b1f4-4856-47da-b909-4eab45492c12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:08.041997 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1042/2055] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:36:07 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 02:36:08.046532 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8980b1f4-4856-47da-b909-4eab45492c12 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:08.047592 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-47789c8a-0c23-4aad-98a4-a284c94e2427 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c99a298a28d49ed9e413d7cc07eb5e2', 'request_id': 'req-47789c8a-0c23-4aad-98a4-a284c94e2427'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:08.054739 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-47789c8a-0c23-4aad-98a4-a284c94e2427 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c99a298a28d49ed9e413d7cc07eb5e2', 'request_id': 'req-47789c8a-0c23-4aad-98a4-a284c94e2427'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:08.062055 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-47789c8a-0c23-4aad-98a4-a284c94e2427 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c99a298a28d49ed9e413d7cc07eb5e2', 'request_id': 'req-47789c8a-0c23-4aad-98a4-a284c94e2427'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:08.115361 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8980b1f4-4856-47da-b909-4eab45492c12 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:08.174688 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-47789c8a-0c23-4aad-98a4-a284c94e2427 admin admin] The token cache is being invalidate because project 4c99a298a28d49ed9e413d7cc07eb5e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 02:36:08.175121 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-47789c8a-0c23-4aad-98a4-a284c94e2427 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:08.175923 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-47789c8a-0c23-4aad-98a4-a284c94e2427 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:08.178936 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1014/2056] 10.222.0.66 () {58 vars in 1232 bytes} [Mon Jul 18 02:36:07 2022] DELETE /identity/v3/projects/4c99a298a28d49ed9e413d7cc07eb5e2 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 02:36:08.186728 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6a625f-32e9-405a-9127-38c49abd6171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:08.186728 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6a625f-32e9-405a-9127-38c49abd6171 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:08.186728 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6a625f-32e9-405a-9127-38c49abd6171 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:08.186728 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6a625f-32e9-405a-9127-38c49abd6171 admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:08.187062 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6a625f-32e9-405a-9127-38c49abd6171 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:08.187062 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6a625f-32e9-405a-9127-38c49abd6171 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:08.190656 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e1c21d-3b68-4d0e-904c-5ed5002edaec None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:08.193364 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e1c21d-3b68-4d0e-904c-5ed5002edaec None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:08.193815 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e1c21d-3b68-4d0e-904c-5ed5002edaec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:08.216468 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-63e1c21d-3b68-4d0e-904c-5ed5002edaec None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:36:08.217659 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-63e1c21d-3b68-4d0e-904c-5ed5002edaec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:08.225713 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1015/2057] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:36:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:36:08.234892 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff2e0aa-f879-46de-ae66-8997851653a5 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:08.235173 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff2e0aa-f879-46de-ae66-8997851653a5 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:08.235455 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff2e0aa-f879-46de-ae66-8997851653a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:08.253553 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-9ff2e0aa-f879-46de-ae66-8997851653a5 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:36:08.254835 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-9ff2e0aa-f879-46de-ae66-8997851653a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:08.263947 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1016/2058] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:36:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 02:36:08.266872 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff2e0aa-f879-46de-ae66-8997851653a5 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:08.271305 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1043/2059] 10.222.0.66 () {60 vars in 1152 bytes} [Mon Jul 18 02:36:08 2022] POST /identity/v3/projects => generated 385 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 02:36:08.271846 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6a625f-32e9-405a-9127-38c49abd6171 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:08.275705 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-ff6a625f-32e9-405a-9127-38c49abd6171 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:08.280127 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff2e0aa-f879-46de-ae66-8997851653a5 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:08.345072 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6a625f-32e9-405a-9127-38c49abd6171 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:08.346913 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f39ec8f-dd0c-47a6-84b4-6896fd457f8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:08.347588 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f39ec8f-dd0c-47a6-84b4-6896fd457f8f admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:08.347907 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f39ec8f-dd0c-47a6-84b4-6896fd457f8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:08.348311 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f39ec8f-dd0c-47a6-84b4-6896fd457f8f admin admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:08.349146 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f39ec8f-dd0c-47a6-84b4-6896fd457f8f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:36:08.349616 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f39ec8f-dd0c-47a6-84b4-6896fd457f8f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:08.349740 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf50c1e1-7f15-41da-86b9-4b10864d034d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:08.350604 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf50c1e1-7f15-41da-86b9-4b10864d034d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:08.350891 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf50c1e1-7f15-41da-86b9-4b10864d034d admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:08.351170 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf50c1e1-7f15-41da-86b9-4b10864d034d admin admin] PATH_INFO: `/v3/projects/a1ba327f33ee42a4bec848dcad35f42a` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:08.351288 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f39ec8f-dd0c-47a6-84b4-6896fd457f8f admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:08.352884 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf50c1e1-7f15-41da-86b9-4b10864d034d admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1ba327f33ee42a4bec848dcad35f42a)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:08.354889 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf50c1e1-7f15-41da-86b9-4b10864d034d admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:08.368703 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1017/2060] 10.222.0.66 () {58 vars in 1152 bytes} [Mon Jul 18 02:36:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 02:36:08.368703 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f39ec8f-dd0c-47a6-84b4-6896fd457f8f admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:08.370188 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-8f39ec8f-dd0c-47a6-84b4-6896fd457f8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:08.382612 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f39ec8f-dd0c-47a6-84b4-6896fd457f8f admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:08.387445 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7252e2-b178-4e6d-8498-909ea2c8fd01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:08.388113 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7252e2-b178-4e6d-8498-909ea2c8fd01 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:08.388459 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7252e2-b178-4e6d-8498-909ea2c8fd01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:08.388740 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7252e2-b178-4e6d-8498-909ea2c8fd01 admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:08.389430 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7252e2-b178-4e6d-8498-909ea2c8fd01 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:08.391384 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7252e2-b178-4e6d-8498-909ea2c8fd01 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:08.488340 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-bf50c1e1-7f15-41da-86b9-4b10864d034d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1ba327f33ee42a4bec848dcad35f42a', 'request_id': 'req-bf50c1e1-7f15-41da-86b9-4b10864d034d'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:08.493887 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-bf50c1e1-7f15-41da-86b9-4b10864d034d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1ba327f33ee42a4bec848dcad35f42a', 'request_id': 'req-bf50c1e1-7f15-41da-86b9-4b10864d034d'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:08.519153 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1018/2061] 10.222.0.66 () {60 vars in 1143 bytes} [Mon Jul 18 02:36:08 2022] POST /identity/v3/users => generated 299 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:36:08.519902 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7252e2-b178-4e6d-8498-909ea2c8fd01 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:08.531081 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7252e2-b178-4e6d-8498-909ea2c8fd01 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:08.573390 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-bf50c1e1-7f15-41da-86b9-4b10864d034d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1ba327f33ee42a4bec848dcad35f42a', 'request_id': 'req-bf50c1e1-7f15-41da-86b9-4b10864d034d'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:08.585916 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-bf50c1e1-7f15-41da-86b9-4b10864d034d admin admin] The token cache is being invalidate because project a1ba327f33ee42a4bec848dcad35f42a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 02:36:08.586243 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-bf50c1e1-7f15-41da-86b9-4b10864d034d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:08.587658 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-bf50c1e1-7f15-41da-86b9-4b10864d034d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:08.589413 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1044/2062] 10.222.0.66 () {58 vars in 1232 bytes} [Mon Jul 18 02:36:08 2022] DELETE /identity/v3/projects/a1ba327f33ee42a4bec848dcad35f42a => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:36:08.592959 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf50c1e1-7f15-41da-86b9-4b10864d034d admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:08.594559 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-bf50c1e1-7f15-41da-86b9-4b10864d034d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:08.598918 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b34821-f722-4f90-b02b-7d334f157bc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:08.599935 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b34821-f722-4f90-b02b-7d334f157bc4 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:08.600366 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b34821-f722-4f90-b02b-7d334f157bc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:08.600718 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b34821-f722-4f90-b02b-7d334f157bc4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:08.601482 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b34821-f722-4f90-b02b-7d334f157bc4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:08.607251 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:DELETE=5,keystone:SELECT=90,keystone:INSERT=5 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:36:08.607513 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b34821-f722-4f90-b02b-7d334f157bc4 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:08.611315 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf50c1e1-7f15-41da-86b9-4b10864d034d admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:08.616653 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4b1b4d-cb03-4c42-8b9e-65cfc3922b0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:08.617239 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4b1b4d-cb03-4c42-8b9e-65cfc3922b0d admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:08.617595 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4b1b4d-cb03-4c42-8b9e-65cfc3922b0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:08.617983 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4b1b4d-cb03-4c42-8b9e-65cfc3922b0d admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:08.619001 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4b1b4d-cb03-4c42-8b9e-65cfc3922b0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:08.620638 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4b1b4d-cb03-4c42-8b9e-65cfc3922b0d admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:08.630895 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1045/2063] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:36:08 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:36:08.634903 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4b1b4d-cb03-4c42-8b9e-65cfc3922b0d admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:08.634903 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-fd4b1b4d-cb03-4c42-8b9e-65cfc3922b0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:08.647674 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4b1b4d-cb03-4c42-8b9e-65cfc3922b0d admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:08.653552 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720ff81c-e378-4664-84b1-2071cdd59a4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:08.654661 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720ff81c-e378-4664-84b1-2071cdd59a4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:08.655018 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720ff81c-e378-4664-84b1-2071cdd59a4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:08.655359 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720ff81c-e378-4664-84b1-2071cdd59a4c admin admin] PATH_INFO: `/v3/users/6e081d54d53f4e94a1829360441519ff` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:08.656728 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720ff81c-e378-4664-84b1-2071cdd59a4c admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e081d54d53f4e94a1829360441519ff)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:08.658929 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720ff81c-e378-4664-84b1-2071cdd59a4c admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:08.698388 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1019/2064] 10.222.0.66 () {60 vars in 1152 bytes} [Mon Jul 18 02:36:08 2022] POST /identity/v3/projects => generated 371 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:36:08.698388 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b34821-f722-4f90-b02b-7d334f157bc4 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:08.756531 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b34821-f722-4f90-b02b-7d334f157bc4 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:08.761377 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c7108b-044d-4534-8e3c-560fb2710f09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:08.762298 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c7108b-044d-4534-8e3c-560fb2710f09 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:08.762617 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c7108b-044d-4534-8e3c-560fb2710f09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:08.762960 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c7108b-044d-4534-8e3c-560fb2710f09 admin admin] PATH_INFO: `/v3/projects/0d93db33796141a2bdc8822baa13b3c7/users/2c7b37634ed9493ebdbb7035e31e7209/roles/d4aa3324f0c84887b41d0dc4acf7f30e` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:08.778161 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c7108b-044d-4534-8e3c-560fb2710f09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d93db33796141a2bdc8822baa13b3c7, user_id=2c7b37634ed9493ebdbb7035e31e7209, role_id=d4aa3324f0c84887b41d0dc4acf7f30e)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:08.780986 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c7108b-044d-4534-8e3c-560fb2710f09 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:08.861354 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1020/2065] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:36:08 2022] PUT /identity/v3/projects/0d93db33796141a2bdc8822baa13b3c7/users/2c7b37634ed9493ebdbb7035e31e7209/roles/d4aa3324f0c84887b41d0dc4acf7f30e => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:36:08.861920 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c7108b-044d-4534-8e3c-560fb2710f09 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:08.863788 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-e6c7108b-044d-4534-8e3c-560fb2710f09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:08.870202 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-720ff81c-e378-4664-84b1-2071cdd59a4c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e081d54d53f4e94a1829360441519ff'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:08.928403 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:36:08.933432 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c7108b-044d-4534-8e3c-560fb2710f09 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:08.938128 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef541b5-8d13-4f28-b0e8-8bfd1f2c74db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:08.938799 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef541b5-8d13-4f28-b0e8-8bfd1f2c74db admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:08.939083 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef541b5-8d13-4f28-b0e8-8bfd1f2c74db admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:08.939360 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef541b5-8d13-4f28-b0e8-8bfd1f2c74db admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:08.940047 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef541b5-8d13-4f28-b0e8-8bfd1f2c74db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:08.941975 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef541b5-8d13-4f28-b0e8-8bfd1f2c74db admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:08.947738 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-720ff81c-e378-4664-84b1-2071cdd59a4c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e081d54d53f4e94a1829360441519ff'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:08.955405 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-720ff81c-e378-4664-84b1-2071cdd59a4c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e081d54d53f4e94a1829360441519ff'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:08.956022 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-720ff81c-e378-4664-84b1-2071cdd59a4c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e081d54d53f4e94a1829360441519ff'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:08.970408 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-720ff81c-e378-4664-84b1-2071cdd59a4c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e081d54d53f4e94a1829360441519ff'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:08.977462 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1046/2066] 10.222.0.66 () {58 vars in 1223 bytes} [Mon Jul 18 02:36:08 2022] DELETE /identity/v3/users/6e081d54d53f4e94a1829360441519ff => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:36:08.978720 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720ff81c-e378-4664-84b1-2071cdd59a4c admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:08.980042 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-720ff81c-e378-4664-84b1-2071cdd59a4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:09.015044 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1021/2067] 10.222.0.66 () {60 vars in 1143 bytes} [Mon Jul 18 02:36:08 2022] POST /identity/v3/users => generated 286 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:36:09.016356 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef541b5-8d13-4f28-b0e8-8bfd1f2c74db admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:09.019152 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-fef541b5-8d13-4f28-b0e8-8bfd1f2c74db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:09.032793 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef541b5-8d13-4f28-b0e8-8bfd1f2c74db admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:09.053725 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720ff81c-e378-4664-84b1-2071cdd59a4c admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:09.061357 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a159fccc-fcd1-41ce-9a6e-af2e17d7a0c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:09.062258 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a159fccc-fcd1-41ce-9a6e-af2e17d7a0c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:09.062404 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a159fccc-fcd1-41ce-9a6e-af2e17d7a0c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:09.062687 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a159fccc-fcd1-41ce-9a6e-af2e17d7a0c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:09.063586 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a159fccc-fcd1-41ce-9a6e-af2e17d7a0c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:09.065217 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a159fccc-fcd1-41ce-9a6e-af2e17d7a0c1 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:09.079311 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1047/2068] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:36:08 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:36:09.080747 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a159fccc-fcd1-41ce-9a6e-af2e17d7a0c1 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:09.091513 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a159fccc-fcd1-41ce-9a6e-af2e17d7a0c1 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:09.096238 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a597959a-865b-4b10-b204-6a8a854e5015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:09.100747 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a597959a-865b-4b10-b204-6a8a854e5015 admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:09.100849 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb84920-3f6f-4957-aaf4-a927fa6a6d6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:09.100956 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a597959a-865b-4b10-b204-6a8a854e5015 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:09.100956 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a597959a-865b-4b10-b204-6a8a854e5015 admin admin] PATH_INFO: `/v3/projects/0d93db33796141a2bdc8822baa13b3c7/users/2c7b37634ed9493ebdbb7035e31e7209/roles/d4aa3324f0c84887b41d0dc4acf7f30e` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:09.101130 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb84920-3f6f-4957-aaf4-a927fa6a6d6a admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:09.101130 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb84920-3f6f-4957-aaf4-a927fa6a6d6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:09.101130 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb84920-3f6f-4957-aaf4-a927fa6a6d6a admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:09.101130 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb84920-3f6f-4957-aaf4-a927fa6a6d6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:09.101424 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a597959a-865b-4b10-b204-6a8a854e5015 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d93db33796141a2bdc8822baa13b3c7, user_id=2c7b37634ed9493ebdbb7035e31e7209, role_id=d4aa3324f0c84887b41d0dc4acf7f30e)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:09.101516 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb84920-3f6f-4957-aaf4-a927fa6a6d6a admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:09.102611 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a597959a-865b-4b10-b204-6a8a854e5015 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:09.111077 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1022/2069] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:36:09 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:36:09.114867 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb84920-3f6f-4957-aaf4-a927fa6a6d6a admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:09.116935 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1048/2070] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:36:09 2022] PUT /identity/v3/projects/0d93db33796141a2bdc8822baa13b3c7/users/2c7b37634ed9493ebdbb7035e31e7209/roles/d4aa3324f0c84887b41d0dc4acf7f30e => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 02:36:09.125368 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb84920-3f6f-4957-aaf4-a927fa6a6d6a admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:09.127198 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f173aa00-ee01-4936-a4bc-d1498f714af9 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:09.127495 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f173aa00-ee01-4936-a4bc-d1498f714af9 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:09.127778 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f173aa00-ee01-4936-a4bc-d1498f714af9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:09.156227 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-f173aa00-ee01-4936-a4bc-d1498f714af9 None None] MFA Rules not processed for user `2c7b37634ed9493ebdbb7035e31e7209`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:36:09.157806 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-f173aa00-ee01-4936-a4bc-d1498f714af9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:09.196936 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba79ea7-715c-4bd5-84e4-0cb75f4e9cff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:09.197047 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba79ea7-715c-4bd5-84e4-0cb75f4e9cff admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:09.197389 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba79ea7-715c-4bd5-84e4-0cb75f4e9cff admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:09.197666 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba79ea7-715c-4bd5-84e4-0cb75f4e9cff admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:09.198565 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba79ea7-715c-4bd5-84e4-0cb75f4e9cff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:09.200260 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba79ea7-715c-4bd5-84e4-0cb75f4e9cff admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:09.211648 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1023/2071] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:36:09 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 02:36:09.215639 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba79ea7-715c-4bd5-84e4-0cb75f4e9cff admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:09.225959 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba79ea7-715c-4bd5-84e4-0cb75f4e9cff admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:09.231542 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fc6815-0c13-40ea-a92c-d5d68a7dad35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:09.232445 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fc6815-0c13-40ea-a92c-d5d68a7dad35 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:09.232727 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fc6815-0c13-40ea-a92c-d5d68a7dad35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:09.233000 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fc6815-0c13-40ea-a92c-d5d68a7dad35 admin admin] PATH_INFO: `/v3/projects/a784b54e7089433cb4918d39f18a9ced/users/5cd0ca9fce844c748c9c562be96f9553/roles/65feabf3b27944a8b01a86c8ac608457` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:09.235691 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1049/2072] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:36:09 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 02:36:09.247384 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f173aa00-ee01-4936-a4bc-d1498f714af9 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:09.248785 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-f173aa00-ee01-4936-a4bc-d1498f714af9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:09.254645 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fc6815-0c13-40ea-a92c-d5d68a7dad35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a784b54e7089433cb4918d39f18a9ced, user_id=5cd0ca9fce844c748c9c562be96f9553, role_id=65feabf3b27944a8b01a86c8ac608457)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:09.257630 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fc6815-0c13-40ea-a92c-d5d68a7dad35 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:09.308652 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f173aa00-ee01-4936-a4bc-d1498f714af9 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:09.312321 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa67099e-d11f-436e-bc9d-81bd4db154b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd49388facdf446885e55b45ef91f01e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:09.312964 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa67099e-d11f-436e-bc9d-81bd4db154b3 service cinder] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:09.313343 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa67099e-d11f-436e-bc9d-81bd4db154b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:09.313647 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa67099e-d11f-436e-bc9d-81bd4db154b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:09.323863 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa67099e-d11f-436e-bc9d-81bd4db154b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:09.325323 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa67099e-d11f-436e-bc9d-81bd4db154b3 service cinder] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:09.333167 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1024/2073] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:36:09 2022] PUT /identity/v3/projects/a784b54e7089433cb4918d39f18a9ced/users/5cd0ca9fce844c748c9c562be96f9553/roles/65feabf3b27944a8b01a86c8ac608457 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 02:36:09.340096 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1050/2074] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:36:09 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:36:09.343709 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1513108-7aa6-4f6c-9de9-095fc86a4a1c None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:09.344026 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1513108-7aa6-4f6c-9de9-095fc86a4a1c None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:09.344312 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1513108-7aa6-4f6c-9de9-095fc86a4a1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:09.370745 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-e1513108-7aa6-4f6c-9de9-095fc86a4a1c None None] MFA Rules not processed for user `5cd0ca9fce844c748c9c562be96f9553`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:36:09.371981 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-e1513108-7aa6-4f6c-9de9-095fc86a4a1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:09.436209 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1025/2075] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:36:09 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:36:09.449745 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa67099e-d11f-436e-bc9d-81bd4db154b3 service cinder] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:09.449933 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1513108-7aa6-4f6c-9de9-095fc86a4a1c None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:09.451369 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-e1513108-7aa6-4f6c-9de9-095fc86a4a1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:09.463056 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-fa67099e-d11f-436e-bc9d-81bd4db154b3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:09.478425 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa67099e-d11f-436e-bc9d-81bd4db154b3 service cinder] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:09.540843 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1513108-7aa6-4f6c-9de9-095fc86a4a1c None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:09.545288 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb45f10-5e70-480a-9aec-3c79f7de9f7c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1391b81e7234bc18e22b9e84f231256', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:09.545940 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb45f10-5e70-480a-9aec-3c79f7de9f7c service glance] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:09.548478 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb45f10-5e70-480a-9aec-3c79f7de9f7c service glance] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:09.548478 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb45f10-5e70-480a-9aec-3c79f7de9f7c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:09.548656 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b147747-ae1a-42db-87b0-1f2352ceb525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:09.549581 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b147747-ae1a-42db-87b0-1f2352ceb525 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:09.549960 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b147747-ae1a-42db-87b0-1f2352ceb525 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:09.550328 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b147747-ae1a-42db-87b0-1f2352ceb525 admin admin] PATH_INFO: `/v3/projects/5138f424cfe342c99531707b76e57e2c` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:09.551522 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b147747-ae1a-42db-87b0-1f2352ceb525 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5138f424cfe342c99531707b76e57e2c)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:09.553498 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b147747-ae1a-42db-87b0-1f2352ceb525 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:09.555909 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb45f10-5e70-480a-9aec-3c79f7de9f7c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:09.557358 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb45f10-5e70-480a-9aec-3c79f7de9f7c service glance] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:09.567190 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1026/2076] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:36:09 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 02:36:09.693449 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-0b147747-ae1a-42db-87b0-1f2352ceb525 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5138f424cfe342c99531707b76e57e2c', 'request_id': 'req-0b147747-ae1a-42db-87b0-1f2352ceb525'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:09.700015 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-0b147747-ae1a-42db-87b0-1f2352ceb525 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5138f424cfe342c99531707b76e57e2c', 'request_id': 'req-0b147747-ae1a-42db-87b0-1f2352ceb525'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:09.774420 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-0b147747-ae1a-42db-87b0-1f2352ceb525 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5138f424cfe342c99531707b76e57e2c', 'request_id': 'req-0b147747-ae1a-42db-87b0-1f2352ceb525'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:09.794067 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-0b147747-ae1a-42db-87b0-1f2352ceb525 admin admin] The token cache is being invalidate because project 5138f424cfe342c99531707b76e57e2c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 02:36:09.794431 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-0b147747-ae1a-42db-87b0-1f2352ceb525 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:09.795104 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-0b147747-ae1a-42db-87b0-1f2352ceb525 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:09.797172 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1051/2077] 10.222.0.66 () {58 vars in 1232 bytes} [Mon Jul 18 02:36:09 2022] DELETE /identity/v3/projects/5138f424cfe342c99531707b76e57e2c => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 02:36:09.802738 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cbd248-619c-432f-b4a5-51242901e18a None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:09.805027 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cbd248-619c-432f-b4a5-51242901e18a None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:09.805027 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cbd248-619c-432f-b4a5-51242901e18a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:09.825747 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-a6cbd248-619c-432f-b4a5-51242901e18a None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:36:09.827094 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-a6cbd248-619c-432f-b4a5-51242901e18a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:09.900657 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1027/2078] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:36:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 02:36:09.902022 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3376193f-bc2f-4a3b-aabe-93da09468cf5 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:09.903009 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3376193f-bc2f-4a3b-aabe-93da09468cf5 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:09.903009 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3376193f-bc2f-4a3b-aabe-93da09468cf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:09.923785 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-3376193f-bc2f-4a3b-aabe-93da09468cf5 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:36:09.925388 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-3376193f-bc2f-4a3b-aabe-93da09468cf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:09.938893 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1052/2079] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:36:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 02:36:09.947487 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cbd248-619c-432f-b4a5-51242901e18a None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:09.957292 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cbd248-619c-432f-b4a5-51242901e18a None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:09.962180 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d00c72-4727-436b-b908-32c493661dab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:09.962848 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d00c72-4727-436b-b908-32c493661dab admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:09.963194 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d00c72-4727-436b-b908-32c493661dab admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:09.963419 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d00c72-4727-436b-b908-32c493661dab admin admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:09.964327 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d00c72-4727-436b-b908-32c493661dab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:36:09.964642 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d00c72-4727-436b-b908-32c493661dab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:09.966274 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d00c72-4727-436b-b908-32c493661dab admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:09.982458 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1028/2080] 10.222.0.66 () {58 vars in 1152 bytes} [Mon Jul 18 02:36:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:36:09.983709 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3376193f-bc2f-4a3b-aabe-93da09468cf5 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:09.993492 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3376193f-bc2f-4a3b-aabe-93da09468cf5 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:09.998302 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b110d9cc-2feb-408c-8b33-915102544371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:09.998959 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b110d9cc-2feb-408c-8b33-915102544371 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:09.999247 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b110d9cc-2feb-408c-8b33-915102544371 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:09.999523 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b110d9cc-2feb-408c-8b33-915102544371 admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:10.000212 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b110d9cc-2feb-408c-8b33-915102544371 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:10.002165 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b110d9cc-2feb-408c-8b33-915102544371 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:10.043996 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:36:10.113071 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:36:10.121858 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1053/2081] 10.222.0.66 () {60 vars in 1152 bytes} [Mon Jul 18 02:36:09 2022] POST /identity/v3/projects => generated 383 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 02:36:10.125311 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d00c72-4727-436b-b908-32c493661dab admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:10.136604 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d00c72-4727-436b-b908-32c493661dab admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:10.192650 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46d8127-1656-46ab-b95e-d1847d4b85ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:10.193318 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46d8127-1656-46ab-b95e-d1847d4b85ef admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:10.193602 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46d8127-1656-46ab-b95e-d1847d4b85ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:10.193878 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46d8127-1656-46ab-b95e-d1847d4b85ef admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:10.194593 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46d8127-1656-46ab-b95e-d1847d4b85ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:10.197098 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46d8127-1656-46ab-b95e-d1847d4b85ef admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:10.284080 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1029/2082] 10.222.0.66 () {60 vars in 1143 bytes} [Mon Jul 18 02:36:10 2022] POST /identity/v3/users => generated 292 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:36:10.288312 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b110d9cc-2feb-408c-8b33-915102544371 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:10.299008 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b110d9cc-2feb-408c-8b33-915102544371 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:10.303614 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3e576f-769d-4f22-8663-d03dbfc9256a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:10.304269 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3e576f-769d-4f22-8663-d03dbfc9256a admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:10.304556 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3e576f-769d-4f22-8663-d03dbfc9256a admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:10.304836 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3e576f-769d-4f22-8663-d03dbfc9256a admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:10.306117 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3e576f-769d-4f22-8663-d03dbfc9256a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:10.307725 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3e576f-769d-4f22-8663-d03dbfc9256a admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:10.318637 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1054/2083] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:36:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:36:10.323113 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46d8127-1656-46ab-b95e-d1847d4b85ef admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:10.333813 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46d8127-1656-46ab-b95e-d1847d4b85ef admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:10.338529 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c526dd-8cd4-42c0-80cb-d896e8b67396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:10.339193 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c526dd-8cd4-42c0-80cb-d896e8b67396 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:10.339482 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c526dd-8cd4-42c0-80cb-d896e8b67396 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:10.339765 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c526dd-8cd4-42c0-80cb-d896e8b67396 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:10.340494 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c526dd-8cd4-42c0-80cb-d896e8b67396 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:10.342133 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c526dd-8cd4-42c0-80cb-d896e8b67396 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:10.352656 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1030/2084] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:36:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:36:10.358393 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3e576f-769d-4f22-8663-d03dbfc9256a admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:10.369061 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3e576f-769d-4f22-8663-d03dbfc9256a admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:10.373711 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d743112-c50e-4921-b050-7b13e0bad8b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:10.374616 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d743112-c50e-4921-b050-7b13e0bad8b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:10.374897 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d743112-c50e-4921-b050-7b13e0bad8b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:10.375170 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d743112-c50e-4921-b050-7b13e0bad8b6 admin admin] PATH_INFO: `/v3/projects/f30256b4abd24f37a048eb5053039845/users/973f5fea84de4c9ea4b806a825f5c533/roles/65feabf3b27944a8b01a86c8ac608457` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:10.390010 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d743112-c50e-4921-b050-7b13e0bad8b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f30256b4abd24f37a048eb5053039845, user_id=973f5fea84de4c9ea4b806a825f5c533, role_id=65feabf3b27944a8b01a86c8ac608457)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:10.392694 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d743112-c50e-4921-b050-7b13e0bad8b6 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:10.479803 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1055/2085] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:36:10 2022] PUT /identity/v3/projects/f30256b4abd24f37a048eb5053039845/users/973f5fea84de4c9ea4b806a825f5c533/roles/65feabf3b27944a8b01a86c8ac608457 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:36:10.493925 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375bdb5d-cc20-4262-955d-6f28c8ade924 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:10.494251 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375bdb5d-cc20-4262-955d-6f28c8ade924 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:10.494678 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375bdb5d-cc20-4262-955d-6f28c8ade924 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:10.521929 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-375bdb5d-cc20-4262-955d-6f28c8ade924 None None] MFA Rules not processed for user `973f5fea84de4c9ea4b806a825f5c533`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:36:10.523173 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-375bdb5d-cc20-4262-955d-6f28c8ade924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:10.570079 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:36:10.602665 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1031/2086] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:36:10 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:36:10.607522 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d743112-c50e-4921-b050-7b13e0bad8b6 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:10.674285 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d743112-c50e-4921-b050-7b13e0bad8b6 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:10.678940 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4b0827-122c-4d05-b844-6c52cdfae1bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:10.679625 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4b0827-122c-4d05-b844-6c52cdfae1bb admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:10.679909 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4b0827-122c-4d05-b844-6c52cdfae1bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:10.680799 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4b0827-122c-4d05-b844-6c52cdfae1bb admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:10.681585 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4b0827-122c-4d05-b844-6c52cdfae1bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:10.683554 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4b0827-122c-4d05-b844-6c52cdfae1bb admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:10.745399 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375bdb5d-cc20-4262-955d-6f28c8ade924 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:10.746924 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-375bdb5d-cc20-4262-955d-6f28c8ade924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:10.761252 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375bdb5d-cc20-4262-955d-6f28c8ade924 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:10.765928 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b59ecbc-5805-4df1-a33a-6db82653604a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:10.766838 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b59ecbc-5805-4df1-a33a-6db82653604a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:10.767158 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b59ecbc-5805-4df1-a33a-6db82653604a admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:10.767436 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b59ecbc-5805-4df1-a33a-6db82653604a admin admin] PATH_INFO: `/v3/users/3c3f3a5269d446cc8a5cc9c54fbd3643` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:10.768895 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b59ecbc-5805-4df1-a33a-6db82653604a admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c3f3a5269d446cc8a5cc9c54fbd3643)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:10.770948 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b59ecbc-5805-4df1-a33a-6db82653604a admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:10.808636 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1056/2087] 10.222.0.66 () {60 vars in 1152 bytes} [Mon Jul 18 02:36:10 2022] POST /identity/v3/projects => generated 385 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 02:36:10.810018 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4b0827-122c-4d05-b844-6c52cdfae1bb admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:10.873589 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4b0827-122c-4d05-b844-6c52cdfae1bb admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:10.878910 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940f5f2d-9fa5-4a95-8204-e583b81d6f94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:10.879591 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940f5f2d-9fa5-4a95-8204-e583b81d6f94 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:10.879877 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940f5f2d-9fa5-4a95-8204-e583b81d6f94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:10.880157 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940f5f2d-9fa5-4a95-8204-e583b81d6f94 admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:10.880890 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940f5f2d-9fa5-4a95-8204-e583b81d6f94 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:10.883219 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940f5f2d-9fa5-4a95-8204-e583b81d6f94 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:10.978059 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:36:10.985539 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1057/2088] 10.222.0.66 () {60 vars in 1143 bytes} [Mon Jul 18 02:36:10 2022] POST /identity/v3/users => generated 293 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 02:36:10.988307 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940f5f2d-9fa5-4a95-8204-e583b81d6f94 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:10.991054 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-3b59ecbc-5805-4df1-a33a-6db82653604a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c3f3a5269d446cc8a5cc9c54fbd3643'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:10.998288 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940f5f2d-9fa5-4a95-8204-e583b81d6f94 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:11.003030 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9520b95a-5206-4b29-9cd0-efa1b99c4f0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:11.003787 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9520b95a-5206-4b29-9cd0-efa1b99c4f0e admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:11.004074 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9520b95a-5206-4b29-9cd0-efa1b99c4f0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:11.004353 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9520b95a-5206-4b29-9cd0-efa1b99c4f0e admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:11.005092 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9520b95a-5206-4b29-9cd0-efa1b99c4f0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:11.006737 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9520b95a-5206-4b29-9cd0-efa1b99c4f0e admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:11.017229 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1058/2089] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:36:10 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 02:36:11.021489 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9520b95a-5206-4b29-9cd0-efa1b99c4f0e admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:11.031348 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9520b95a-5206-4b29-9cd0-efa1b99c4f0e admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:11.036021 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371e50b0-a6b0-4504-869f-981151b40892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:11.036640 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371e50b0-a6b0-4504-869f-981151b40892 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:11.036923 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371e50b0-a6b0-4504-869f-981151b40892 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:11.037198 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371e50b0-a6b0-4504-869f-981151b40892 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:11.037890 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371e50b0-a6b0-4504-869f-981151b40892 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:11.039547 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371e50b0-a6b0-4504-869f-981151b40892 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:11.049922 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1059/2090] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:36:11 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 02:36:11.055761 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371e50b0-a6b0-4504-869f-981151b40892 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:11.065013 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371e50b0-a6b0-4504-869f-981151b40892 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:11.069508 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236d42a9-0703-4c34-b494-04d0508ddc1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:11.070366 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236d42a9-0703-4c34-b494-04d0508ddc1b admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:11.071290 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236d42a9-0703-4c34-b494-04d0508ddc1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:11.071669 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236d42a9-0703-4c34-b494-04d0508ddc1b admin admin] PATH_INFO: `/v3/projects/be6d2e1b8555409ebb608e3f6a5e7ad6/users/ae77aa9557b1433ea7681caffd85b634/roles/65feabf3b27944a8b01a86c8ac608457` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:11.086742 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236d42a9-0703-4c34-b494-04d0508ddc1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=be6d2e1b8555409ebb608e3f6a5e7ad6, user_id=ae77aa9557b1433ea7681caffd85b634, role_id=65feabf3b27944a8b01a86c8ac608457)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:11.089371 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236d42a9-0703-4c34-b494-04d0508ddc1b admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:11.110190 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-3b59ecbc-5805-4df1-a33a-6db82653604a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c3f3a5269d446cc8a5cc9c54fbd3643'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:11.110867 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-3b59ecbc-5805-4df1-a33a-6db82653604a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c3f3a5269d446cc8a5cc9c54fbd3643'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:11.124157 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-3b59ecbc-5805-4df1-a33a-6db82653604a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c3f3a5269d446cc8a5cc9c54fbd3643'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:11.124822 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-3b59ecbc-5805-4df1-a33a-6db82653604a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c3f3a5269d446cc8a5cc9c54fbd3643'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:11.137187 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1032/2091] 10.222.0.66 () {58 vars in 1223 bytes} [Mon Jul 18 02:36:10 2022] DELETE /identity/v3/users/3c3f3a5269d446cc8a5cc9c54fbd3643 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:36:11.139499 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:36:11.141664 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b485ec17-ea7d-4b94-961f-4834e861920e None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:11.141952 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b485ec17-ea7d-4b94-961f-4834e861920e None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:11.142267 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b485ec17-ea7d-4b94-961f-4834e861920e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:11.160566 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-b485ec17-ea7d-4b94-961f-4834e861920e None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:36:11.161754 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-b485ec17-ea7d-4b94-961f-4834e861920e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:11.198104 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1060/2092] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:36:11 2022] PUT /identity/v3/projects/be6d2e1b8555409ebb608e3f6a5e7ad6/users/ae77aa9557b1433ea7681caffd85b634/roles/65feabf3b27944a8b01a86c8ac608457 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 02:36:11.214421 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b84d67-147b-41de-87c3-5dfa4f58d07f None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:11.214710 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b84d67-147b-41de-87c3-5dfa4f58d07f None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:11.214989 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b84d67-147b-41de-87c3-5dfa4f58d07f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:11.220207 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1033/2093] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:36:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 02:36:11.230545 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b485ec17-ea7d-4b94-961f-4834e861920e None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:11.231927 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-b485ec17-ea7d-4b94-961f-4834e861920e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:11.241514 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-45b84d67-147b-41de-87c3-5dfa4f58d07f None None] MFA Rules not processed for user `ae77aa9557b1433ea7681caffd85b634`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:36:11.243382 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-45b84d67-147b-41de-87c3-5dfa4f58d07f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:11.288243 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b485ec17-ea7d-4b94-961f-4834e861920e None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:11.306474 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1061/2094] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:36:11 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 02:36:11.311108 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b84d67-147b-41de-87c3-5dfa4f58d07f None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:11.312498 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-45b84d67-147b-41de-87c3-5dfa4f58d07f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:11.344966 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195a32e1-783e-4eae-a6da-c4e75fb802f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eaf19a259c2741cea375a38763e33cdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:11.345627 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195a32e1-783e-4eae-a6da-c4e75fb802f5 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:11.345909 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195a32e1-783e-4eae-a6da-c4e75fb802f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:11.346294 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195a32e1-783e-4eae-a6da-c4e75fb802f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:11.378176 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b84d67-147b-41de-87c3-5dfa4f58d07f None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:11.383386 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc093c65-acf5-4614-ac9e-c62ba874e181 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:11.384156 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc093c65-acf5-4614-ac9e-c62ba874e181 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:11.384444 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc093c65-acf5-4614-ac9e-c62ba874e181 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:11.384721 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc093c65-acf5-4614-ac9e-c62ba874e181 admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:11.385420 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc093c65-acf5-4614-ac9e-c62ba874e181 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:11.387385 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc093c65-acf5-4614-ac9e-c62ba874e181 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:11.389439 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:36:11.410470 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195a32e1-783e-4eae-a6da-c4e75fb802f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:11.411926 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195a32e1-783e-4eae-a6da-c4e75fb802f5 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:11.422244 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1034/2095] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:36:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 02:36:11.481673 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1062/2096] 10.222.0.66 () {60 vars in 1152 bytes} [Mon Jul 18 02:36:11 2022] POST /identity/v3/projects => generated 385 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 02:36:11.484945 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195a32e1-783e-4eae-a6da-c4e75fb802f5 service neutron] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:11.492819 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195a32e1-783e-4eae-a6da-c4e75fb802f5 service neutron] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:11.493758 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:36:11.551319 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6044b5f-1566-43f5-8b7a-8b2e42f5c95a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:11.552020 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6044b5f-1566-43f5-8b7a-8b2e42f5c95a admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:11.552305 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6044b5f-1566-43f5-8b7a-8b2e42f5c95a admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:11.552582 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6044b5f-1566-43f5-8b7a-8b2e42f5c95a admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:11.553274 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6044b5f-1566-43f5-8b7a-8b2e42f5c95a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:11.555261 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6044b5f-1566-43f5-8b7a-8b2e42f5c95a admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:11.637257 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1035/2097] 10.222.0.66 () {60 vars in 1143 bytes} [Mon Jul 18 02:36:11 2022] POST /identity/v3/users => generated 299 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:36:11.641283 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc093c65-acf5-4614-ac9e-c62ba874e181 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:11.654977 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc093c65-acf5-4614-ac9e-c62ba874e181 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:11.660034 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37412a84-5c8d-48a0-aeec-60ba75fd7cdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:11.660712 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37412a84-5c8d-48a0-aeec-60ba75fd7cdd admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:11.661000 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37412a84-5c8d-48a0-aeec-60ba75fd7cdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:11.661281 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37412a84-5c8d-48a0-aeec-60ba75fd7cdd admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:11.663315 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37412a84-5c8d-48a0-aeec-60ba75fd7cdd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:11.663758 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37412a84-5c8d-48a0-aeec-60ba75fd7cdd admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:11.675418 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1063/2098] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:36:11 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 02:36:11.680032 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6044b5f-1566-43f5-8b7a-8b2e42f5c95a admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:11.691215 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6044b5f-1566-43f5-8b7a-8b2e42f5c95a admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:11.696189 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a6c940-c2ff-4d97-818c-f8b36b60cc69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:11.697082 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a6c940-c2ff-4d97-818c-f8b36b60cc69 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:11.697431 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a6c940-c2ff-4d97-818c-f8b36b60cc69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:11.697906 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a6c940-c2ff-4d97-818c-f8b36b60cc69 admin admin] PATH_INFO: `/v3/projects/17985eb401a34a2ba5c45ce32ba39332/users/97473baa34c34f25920c7f9d874eb4e8/roles/d4aa3324f0c84887b41d0dc4acf7f30e` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:11.714195 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a6c940-c2ff-4d97-818c-f8b36b60cc69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17985eb401a34a2ba5c45ce32ba39332, user_id=97473baa34c34f25920c7f9d874eb4e8, role_id=d4aa3324f0c84887b41d0dc4acf7f30e)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:11.716872 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a6c940-c2ff-4d97-818c-f8b36b60cc69 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:11.815490 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1036/2099] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:36:11 2022] PUT /identity/v3/projects/17985eb401a34a2ba5c45ce32ba39332/users/97473baa34c34f25920c7f9d874eb4e8/roles/d4aa3324f0c84887b41d0dc4acf7f30e => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 02:36:11.819521 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37412a84-5c8d-48a0-aeec-60ba75fd7cdd admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:11.830019 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37412a84-5c8d-48a0-aeec-60ba75fd7cdd admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:11.884966 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5725b2a-f5f9-4081-ab1e-e22c107df593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:11.885632 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5725b2a-f5f9-4081-ab1e-e22c107df593 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:11.885916 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5725b2a-f5f9-4081-ab1e-e22c107df593 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:11.886229 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5725b2a-f5f9-4081-ab1e-e22c107df593 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:11.886947 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5725b2a-f5f9-4081-ab1e-e22c107df593 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:11.888583 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5725b2a-f5f9-4081-ab1e-e22c107df593 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:11.900834 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1064/2100] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:36:11 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:36:11.910152 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5725b2a-f5f9-4081-ab1e-e22c107df593 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:11.910748 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a6c940-c2ff-4d97-818c-f8b36b60cc69 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:11.912714 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-a2a6c940-c2ff-4d97-818c-f8b36b60cc69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:11.920971 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5725b2a-f5f9-4081-ab1e-e22c107df593 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:11.926432 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:36:11.927511 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf70e491-11a3-4b2b-9c87-ac16944687ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:11.929040 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf70e491-11a3-4b2b-9c87-ac16944687ce admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:11.929040 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf70e491-11a3-4b2b-9c87-ac16944687ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:11.929040 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf70e491-11a3-4b2b-9c87-ac16944687ce admin admin] PATH_INFO: `/v3/projects/17985eb401a34a2ba5c45ce32ba39332/users/97473baa34c34f25920c7f9d874eb4e8/roles/d4aa3324f0c84887b41d0dc4acf7f30e` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:11.930509 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a6c940-c2ff-4d97-818c-f8b36b60cc69 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:11.931193 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf70e491-11a3-4b2b-9c87-ac16944687ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=17985eb401a34a2ba5c45ce32ba39332, user_id=97473baa34c34f25920c7f9d874eb4e8, role_id=d4aa3324f0c84887b41d0dc4acf7f30e)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:11.934027 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf70e491-11a3-4b2b-9c87-ac16944687ce admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:11.935440 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ee2f93-f1c2-454d-a96c-f8b9b262f0da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:11.936369 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ee2f93-f1c2-454d-a96c-f8b9b262f0da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:11.936659 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ee2f93-f1c2-454d-a96c-f8b9b262f0da admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:11.936939 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ee2f93-f1c2-454d-a96c-f8b9b262f0da admin admin] PATH_INFO: `/v3/projects/5477a15859d44821b12789b6dbd124ad` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:11.938164 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ee2f93-f1c2-454d-a96c-f8b9b262f0da admin admin] RBAC: Authorizing `identity:delete_project(project_id=5477a15859d44821b12789b6dbd124ad)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:11.940165 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ee2f93-f1c2-454d-a96c-f8b9b262f0da admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:11.944890 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1065/2101] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:36:11 2022] PUT /identity/v3/projects/17985eb401a34a2ba5c45ce32ba39332/users/97473baa34c34f25920c7f9d874eb4e8/roles/d4aa3324f0c84887b41d0dc4acf7f30e => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 02:36:11.956862 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ac7cba-6715-4f00-adcb-aa47ab1f3c33 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:11.957152 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ac7cba-6715-4f00-adcb-aa47ab1f3c33 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:11.957428 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ac7cba-6715-4f00-adcb-aa47ab1f3c33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:11.989247 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-e1ac7cba-6715-4f00-adcb-aa47ab1f3c33 None None] MFA Rules not processed for user `97473baa34c34f25920c7f9d874eb4e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:36:11.990291 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-e1ac7cba-6715-4f00-adcb-aa47ab1f3c33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:12.057383 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-14ee2f93-f1c2-454d-a96c-f8b9b262f0da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5477a15859d44821b12789b6dbd124ad', 'request_id': 'req-14ee2f93-f1c2-454d-a96c-f8b9b262f0da'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:12.065983 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-14ee2f93-f1c2-454d-a96c-f8b9b262f0da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5477a15859d44821b12789b6dbd124ad', 'request_id': 'req-14ee2f93-f1c2-454d-a96c-f8b9b262f0da'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:12.068781 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1066/2102] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:36:11 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 02:36:12.074481 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-14ee2f93-f1c2-454d-a96c-f8b9b262f0da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5477a15859d44821b12789b6dbd124ad', 'request_id': 'req-14ee2f93-f1c2-454d-a96c-f8b9b262f0da'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:12.080841 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ac7cba-6715-4f00-adcb-aa47ab1f3c33 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:12.083724 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-e1ac7cba-6715-4f00-adcb-aa47ab1f3c33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:12.102874 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:36:12.142361 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ac7cba-6715-4f00-adcb-aa47ab1f3c33 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:12.155866 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-14ee2f93-f1c2-454d-a96c-f8b9b262f0da admin admin] The token cache is being invalidate because project 5477a15859d44821b12789b6dbd124ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 02:36:12.156154 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-14ee2f93-f1c2-454d-a96c-f8b9b262f0da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:12.156827 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-14ee2f93-f1c2-454d-a96c-f8b9b262f0da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:12.158512 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1037/2103] 10.222.0.66 () {58 vars in 1232 bytes} [Mon Jul 18 02:36:11 2022] DELETE /identity/v3/projects/5477a15859d44821b12789b6dbd124ad => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:36:12.162309 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ee2f93-f1c2-454d-a96c-f8b9b262f0da admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:12.163652 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-14ee2f93-f1c2-454d-a96c-f8b9b262f0da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:12.191083 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c308b1d-00ad-4fad-9f79-743a38f50fa4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd49388facdf446885e55b45ef91f01e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:12.191740 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c308b1d-00ad-4fad-9f79-743a38f50fa4 service cinder] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:12.192024 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c308b1d-00ad-4fad-9f79-743a38f50fa4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:12.192296 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c308b1d-00ad-4fad-9f79-743a38f50fa4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:12.194160 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-9c308b1d-00ad-4fad-9f79-743a38f50fa4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:12.229572 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ee2f93-f1c2-454d-a96c-f8b9b262f0da admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:12.234289 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309ee45a-a99d-4bc0-9e8f-88cd23548c71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:12.235157 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309ee45a-a99d-4bc0-9e8f-88cd23548c71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:12.235441 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309ee45a-a99d-4bc0-9e8f-88cd23548c71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:12.235719 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309ee45a-a99d-4bc0-9e8f-88cd23548c71 admin admin] PATH_INFO: `/v3/users/2c7b37634ed9493ebdbb7035e31e7209` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:12.236932 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309ee45a-a99d-4bc0-9e8f-88cd23548c71 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c7b37634ed9493ebdbb7035e31e7209)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:12.238907 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309ee45a-a99d-4bc0-9e8f-88cd23548c71 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:12.254062 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c308b1d-00ad-4fad-9f79-743a38f50fa4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:12.255542 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c308b1d-00ad-4fad-9f79-743a38f50fa4 service cinder] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:12.265341 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1067/2104] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:36:12 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 02:36:12.429093 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-309ee45a-a99d-4bc0-9e8f-88cd23548c71 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c7b37634ed9493ebdbb7035e31e7209'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:12.524945 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-309ee45a-a99d-4bc0-9e8f-88cd23548c71 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c7b37634ed9493ebdbb7035e31e7209'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:12.526647 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:36:12.529784 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-309ee45a-a99d-4bc0-9e8f-88cd23548c71 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c7b37634ed9493ebdbb7035e31e7209'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:12.541036 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-309ee45a-a99d-4bc0-9e8f-88cd23548c71 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c7b37634ed9493ebdbb7035e31e7209'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:12.541714 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-309ee45a-a99d-4bc0-9e8f-88cd23548c71 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c7b37634ed9493ebdbb7035e31e7209'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:12.559553 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1038/2105] 10.222.0.66 () {58 vars in 1223 bytes} [Mon Jul 18 02:36:12 2022] DELETE /identity/v3/users/2c7b37634ed9493ebdbb7035e31e7209 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 02:36:12.697240 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:36:12.809114 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:36:13.370369 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c308b1d-00ad-4fad-9f79-743a38f50fa4 service cinder] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:13.381608 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c308b1d-00ad-4fad-9f79-743a38f50fa4 service cinder] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:13.385550 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:36:13.511492 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c09d8a-8d29-4b4a-9799-5b207ca76879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:13.512563 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c09d8a-8d29-4b4a-9799-5b207ca76879 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:13.512874 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c09d8a-8d29-4b4a-9799-5b207ca76879 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:13.513133 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c09d8a-8d29-4b4a-9799-5b207ca76879 admin admin] PATH_INFO: `/v3/projects/0d93db33796141a2bdc8822baa13b3c7` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:13.514444 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c09d8a-8d29-4b4a-9799-5b207ca76879 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d93db33796141a2bdc8822baa13b3c7)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:13.516410 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c09d8a-8d29-4b4a-9799-5b207ca76879 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:13.650148 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-75c09d8a-8d29-4b4a-9799-5b207ca76879 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d93db33796141a2bdc8822baa13b3c7', 'request_id': 'req-75c09d8a-8d29-4b4a-9799-5b207ca76879'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:13.656661 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-75c09d8a-8d29-4b4a-9799-5b207ca76879 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d93db33796141a2bdc8822baa13b3c7', 'request_id': 'req-75c09d8a-8d29-4b4a-9799-5b207ca76879'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:13.734266 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-75c09d8a-8d29-4b4a-9799-5b207ca76879 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d93db33796141a2bdc8822baa13b3c7', 'request_id': 'req-75c09d8a-8d29-4b4a-9799-5b207ca76879'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:13.749971 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-75c09d8a-8d29-4b4a-9799-5b207ca76879 admin admin] The token cache is being invalidate because project 0d93db33796141a2bdc8822baa13b3c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 02:36:13.750293 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-75c09d8a-8d29-4b4a-9799-5b207ca76879 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:13.750970 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-75c09d8a-8d29-4b4a-9799-5b207ca76879 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:13.756122 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1068/2106] 10.222.0.66 () {58 vars in 1232 bytes} [Mon Jul 18 02:36:13 2022] DELETE /identity/v3/projects/0d93db33796141a2bdc8822baa13b3c7 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:36:13.758666 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c160d472-ed6c-4030-801f-4880476628fd None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:13.758981 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c160d472-ed6c-4030-801f-4880476628fd None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:13.759260 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c160d472-ed6c-4030-801f-4880476628fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:13.779643 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-c160d472-ed6c-4030-801f-4880476628fd None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:36:13.780890 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-c160d472-ed6c-4030-801f-4880476628fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:13.842262 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1039/2107] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:36:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:36:13.854807 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9dcb5f-c463-4107-9a79-85fdf5f33d13 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:13.854807 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9dcb5f-c463-4107-9a79-85fdf5f33d13 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:13.854807 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9dcb5f-c463-4107-9a79-85fdf5f33d13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:13.878173 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-bb9dcb5f-c463-4107-9a79-85fdf5f33d13 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:36:13.879389 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-bb9dcb5f-c463-4107-9a79-85fdf5f33d13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:13.890308 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1069/2108] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:36:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:36:13.896023 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c160d472-ed6c-4030-801f-4880476628fd None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:13.908639 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c160d472-ed6c-4030-801f-4880476628fd None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:13.915835 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f120c932-51aa-42b8-9fcc-42343651c888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:13.917360 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f120c932-51aa-42b8-9fcc-42343651c888 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:13.917360 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f120c932-51aa-42b8-9fcc-42343651c888 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:13.917360 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f120c932-51aa-42b8-9fcc-42343651c888 admin admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:13.917809 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f120c932-51aa-42b8-9fcc-42343651c888 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:36:13.918146 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f120c932-51aa-42b8-9fcc-42343651c888 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:13.919784 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f120c932-51aa-42b8-9fcc-42343651c888 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:13.932413 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1040/2109] 10.222.0.66 () {58 vars in 1152 bytes} [Mon Jul 18 02:36:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:36:13.936688 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9dcb5f-c463-4107-9a79-85fdf5f33d13 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:13.945875 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9dcb5f-c463-4107-9a79-85fdf5f33d13 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:13.950787 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e213ef-6796-4f99-85ba-86b010672e03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:13.951452 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e213ef-6796-4f99-85ba-86b010672e03 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:13.951739 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e213ef-6796-4f99-85ba-86b010672e03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:13.952046 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e213ef-6796-4f99-85ba-86b010672e03 admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:13.952757 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e213ef-6796-4f99-85ba-86b010672e03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:13.954782 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e213ef-6796-4f99-85ba-86b010672e03 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:14.051259 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1070/2110] 10.222.0.66 () {60 vars in 1152 bytes} [Mon Jul 18 02:36:13 2022] POST /identity/v3/projects => generated 361 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:36:14.055393 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f120c932-51aa-42b8-9fcc-42343651c888 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:14.074506 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f120c932-51aa-42b8-9fcc-42343651c888 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:14.153100 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:36:14.178439 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57531c2-6446-4de4-bd83-fa102928f5e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:14.179143 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57531c2-6446-4de4-bd83-fa102928f5e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:14.179414 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57531c2-6446-4de4-bd83-fa102928f5e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:14.179735 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57531c2-6446-4de4-bd83-fa102928f5e2 admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:14.180445 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57531c2-6446-4de4-bd83-fa102928f5e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:14.182609 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57531c2-6446-4de4-bd83-fa102928f5e2 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:14.255653 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1041/2111] 10.222.0.66 () {60 vars in 1143 bytes} [Mon Jul 18 02:36:14 2022] POST /identity/v3/users => generated 281 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:36:14.260417 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e213ef-6796-4f99-85ba-86b010672e03 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:14.271843 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e213ef-6796-4f99-85ba-86b010672e03 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:14.279097 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e954d4a-ec24-4c4c-9a7e-8207346564c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:14.279807 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e954d4a-ec24-4c4c-9a7e-8207346564c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:14.280098 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e954d4a-ec24-4c4c-9a7e-8207346564c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:14.280378 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e954d4a-ec24-4c4c-9a7e-8207346564c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:14.281111 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e954d4a-ec24-4c4c-9a7e-8207346564c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:14.282835 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e954d4a-ec24-4c4c-9a7e-8207346564c0 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:14.295019 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1071/2112] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:36:14 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:36:14.298900 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57531c2-6446-4de4-bd83-fa102928f5e2 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:14.309578 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57531c2-6446-4de4-bd83-fa102928f5e2 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:14.315607 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4f05ac-5c0e-4e3f-8206-f1c15f4d6205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:14.316574 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4f05ac-5c0e-4e3f-8206-f1c15f4d6205 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:14.322201 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4f05ac-5c0e-4e3f-8206-f1c15f4d6205 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:14.322201 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4f05ac-5c0e-4e3f-8206-f1c15f4d6205 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:14.322201 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4f05ac-5c0e-4e3f-8206-f1c15f4d6205 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:14.322201 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4f05ac-5c0e-4e3f-8206-f1c15f4d6205 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:14.332370 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1042/2113] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:36:14 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:36:14.337597 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e954d4a-ec24-4c4c-9a7e-8207346564c0 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:14.357225 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e954d4a-ec24-4c4c-9a7e-8207346564c0 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:14.366532 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b0fa01-463e-4e74-9dca-4ba3075b3bd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:14.367818 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b0fa01-463e-4e74-9dca-4ba3075b3bd0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:14.368165 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b0fa01-463e-4e74-9dca-4ba3075b3bd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:14.368451 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b0fa01-463e-4e74-9dca-4ba3075b3bd0 admin admin] PATH_INFO: `/v3/projects/d83931d680f4471da846545a630ad353/users/e1649b79d3404d13abafc71daa237869/roles/65feabf3b27944a8b01a86c8ac608457` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:14.394154 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b0fa01-463e-4e74-9dca-4ba3075b3bd0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d83931d680f4471da846545a630ad353, user_id=e1649b79d3404d13abafc71daa237869, role_id=65feabf3b27944a8b01a86c8ac608457)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:14.399701 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b0fa01-463e-4e74-9dca-4ba3075b3bd0 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:14.495186 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:36:14.540859 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1072/2114] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:36:14 2022] PUT /identity/v3/projects/d83931d680f4471da846545a630ad353/users/e1649b79d3404d13abafc71daa237869/roles/65feabf3b27944a8b01a86c8ac608457 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:36:14.552128 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fbd46b-605f-4e11-ba60-749a631128e9 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:14.552425 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fbd46b-605f-4e11-ba60-749a631128e9 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:14.552706 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fbd46b-605f-4e11-ba60-749a631128e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:14.579750 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-43fbd46b-605f-4e11-ba60-749a631128e9 None None] MFA Rules not processed for user `e1649b79d3404d13abafc71daa237869`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:36:14.580977 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-43fbd46b-605f-4e11-ba60-749a631128e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:14.647486 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1043/2115] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:36:14 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:36:14.839366 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b0fa01-463e-4e74-9dca-4ba3075b3bd0 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:14.841138 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-91b0fa01-463e-4e74-9dca-4ba3075b3bd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:14.901919 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b0fa01-463e-4e74-9dca-4ba3075b3bd0 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:14.905761 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab96cf8e-58a7-4a63-a48f-b0a3b826b11f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd49388facdf446885e55b45ef91f01e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:14.906447 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab96cf8e-58a7-4a63-a48f-b0a3b826b11f service cinder] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:14.906738 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab96cf8e-58a7-4a63-a48f-b0a3b826b11f service cinder] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:14.907139 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab96cf8e-58a7-4a63-a48f-b0a3b826b11f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:14.917213 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab96cf8e-58a7-4a63-a48f-b0a3b826b11f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:14.918773 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab96cf8e-58a7-4a63-a48f-b0a3b826b11f service cinder] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:14.930320 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1073/2116] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:36:14 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:36:14.943422 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fbd46b-605f-4e11-ba60-749a631128e9 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:15.006917 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fbd46b-605f-4e11-ba60-749a631128e9 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:15.011589 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe40859-c067-49e5-9653-6cc6ee4bfb27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:15.012400 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe40859-c067-49e5-9653-6cc6ee4bfb27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:15.012689 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe40859-c067-49e5-9653-6cc6ee4bfb27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:15.012970 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe40859-c067-49e5-9653-6cc6ee4bfb27 admin admin] PATH_INFO: `/v3/users/e1649b79d3404d13abafc71daa237869` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:15.014191 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe40859-c067-49e5-9653-6cc6ee4bfb27 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1649b79d3404d13abafc71daa237869)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:15.016153 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe40859-c067-49e5-9653-6cc6ee4bfb27 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:15.102123 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:36:15.141093 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:36:15.232491 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:36:15.245972 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-8fe40859-c067-49e5-9653-6cc6ee4bfb27 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1649b79d3404d13abafc71daa237869'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:15.294975 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-8fe40859-c067-49e5-9653-6cc6ee4bfb27 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1649b79d3404d13abafc71daa237869'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:15.295622 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-8fe40859-c067-49e5-9653-6cc6ee4bfb27 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1649b79d3404d13abafc71daa237869'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:15.308298 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-8fe40859-c067-49e5-9653-6cc6ee4bfb27 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1649b79d3404d13abafc71daa237869'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:15.309153 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-8fe40859-c067-49e5-9653-6cc6ee4bfb27 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1649b79d3404d13abafc71daa237869'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:15.322173 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1044/2117] 10.222.0.66 () {58 vars in 1223 bytes} [Mon Jul 18 02:36:14 2022] DELETE /identity/v3/users/e1649b79d3404d13abafc71daa237869 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:36:15.326439 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdadce53-92ea-4901-9f15-7d8543f8cdc7 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:15.326730 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdadce53-92ea-4901-9f15-7d8543f8cdc7 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:15.329796 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdadce53-92ea-4901-9f15-7d8543f8cdc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:15.345633 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-bdadce53-92ea-4901-9f15-7d8543f8cdc7 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:36:15.352419 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-bdadce53-92ea-4901-9f15-7d8543f8cdc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:15.402748 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe40859-c067-49e5-9653-6cc6ee4bfb27 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:15.404160 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-8fe40859-c067-49e5-9653-6cc6ee4bfb27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:15.408658 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1074/2118] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:36:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:36:15.419082 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdadce53-92ea-4901-9f15-7d8543f8cdc7 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:15.420477 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-bdadce53-92ea-4901-9f15-7d8543f8cdc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:15.462885 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe40859-c067-49e5-9653-6cc6ee4bfb27 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:15.467173 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3647036b-e9c7-4c7a-8690-ae2a96e7c0ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd49388facdf446885e55b45ef91f01e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:15.467833 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3647036b-e9c7-4c7a-8690-ae2a96e7c0ad service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:15.468483 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3647036b-e9c7-4c7a-8690-ae2a96e7c0ad service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:15.468483 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3647036b-e9c7-4c7a-8690-ae2a96e7c0ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:15.470391 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-3647036b-e9c7-4c7a-8690-ae2a96e7c0ad service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:15.487029 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdadce53-92ea-4901-9f15-7d8543f8cdc7 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:15.489944 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d87254-bd0a-4175-943b-be26f684c920 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eaf19a259c2741cea375a38763e33cdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:15.490620 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d87254-bd0a-4175-943b-be26f684c920 service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:15.490901 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d87254-bd0a-4175-943b-be26f684c920 service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:15.491177 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d87254-bd0a-4175-943b-be26f684c920 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:15.503757 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d87254-bd0a-4175-943b-be26f684c920 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:15.507183 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d87254-bd0a-4175-943b-be26f684c920 service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:15.520572 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1075/2119] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:36:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 02:36:15.543185 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:36:15.545525 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3647036b-e9c7-4c7a-8690-ae2a96e7c0ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:15.547306 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3647036b-e9c7-4c7a-8690-ae2a96e7c0ad service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:15.562995 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1045/2120] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:36:15 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 02:36:16.048052 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d87254-bd0a-4175-943b-be26f684c920 service neutron] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:16.048052 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-13d87254-bd0a-4175-943b-be26f684c920 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:16.065023 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d87254-bd0a-4175-943b-be26f684c920 service neutron] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:16.073380 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0668eb23-cf20-4218-88f3-20c81ac19558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:16.076257 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0668eb23-cf20-4218-88f3-20c81ac19558 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:16.076257 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0668eb23-cf20-4218-88f3-20c81ac19558 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:16.076257 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0668eb23-cf20-4218-88f3-20c81ac19558 admin admin] PATH_INFO: `/v3/projects/d83931d680f4471da846545a630ad353` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:16.077081 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0668eb23-cf20-4218-88f3-20c81ac19558 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d83931d680f4471da846545a630ad353)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:16.078417 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0668eb23-cf20-4218-88f3-20c81ac19558 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:16.146863 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:36:16.192004 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-0668eb23-cf20-4218-88f3-20c81ac19558 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd83931d680f4471da846545a630ad353', 'request_id': 'req-0668eb23-cf20-4218-88f3-20c81ac19558'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:16.197730 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-0668eb23-cf20-4218-88f3-20c81ac19558 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd83931d680f4471da846545a630ad353', 'request_id': 'req-0668eb23-cf20-4218-88f3-20c81ac19558'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:16.298802 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-0668eb23-cf20-4218-88f3-20c81ac19558 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd83931d680f4471da846545a630ad353', 'request_id': 'req-0668eb23-cf20-4218-88f3-20c81ac19558'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:16.308255 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-0668eb23-cf20-4218-88f3-20c81ac19558 admin admin] The token cache is being invalidate because project d83931d680f4471da846545a630ad353 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 02:36:16.308548 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-0668eb23-cf20-4218-88f3-20c81ac19558 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:16.309214 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-0668eb23-cf20-4218-88f3-20c81ac19558 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:16.311162 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1076/2121] 10.222.0.66 () {58 vars in 1232 bytes} [Mon Jul 18 02:36:16 2022] DELETE /identity/v3/projects/d83931d680f4471da846545a630ad353 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:36:16.315866 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe71f76-5431-4da6-a1f8-4c765a592c42 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:16.316397 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe71f76-5431-4da6-a1f8-4c765a592c42 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:16.316753 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe71f76-5431-4da6-a1f8-4c765a592c42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:16.336679 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-efe71f76-5431-4da6-a1f8-4c765a592c42 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:36:16.337989 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-efe71f76-5431-4da6-a1f8-4c765a592c42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:16.396689 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1046/2122] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:36:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:36:16.405260 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84555a4d-bed8-4bcc-9af3-350ae4de2f54 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:16.405546 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84555a4d-bed8-4bcc-9af3-350ae4de2f54 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:16.411380 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84555a4d-bed8-4bcc-9af3-350ae4de2f54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:16.427310 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-84555a4d-bed8-4bcc-9af3-350ae4de2f54 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:36:16.428522 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-84555a4d-bed8-4bcc-9af3-350ae4de2f54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:16.436705 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1077/2123] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:36:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:36:16.440993 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe71f76-5431-4da6-a1f8-4c765a592c42 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:16.451247 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe71f76-5431-4da6-a1f8-4c765a592c42 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:16.455907 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df66b887-6508-4fa2-bb0b-2ea745876b3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:16.456555 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df66b887-6508-4fa2-bb0b-2ea745876b3c admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:16.456842 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df66b887-6508-4fa2-bb0b-2ea745876b3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:16.457124 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df66b887-6508-4fa2-bb0b-2ea745876b3c admin admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:16.457852 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df66b887-6508-4fa2-bb0b-2ea745876b3c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:36:16.458198 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df66b887-6508-4fa2-bb0b-2ea745876b3c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:16.459827 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df66b887-6508-4fa2-bb0b-2ea745876b3c admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:16.472063 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1047/2124] 10.222.0.66 () {58 vars in 1152 bytes} [Mon Jul 18 02:36:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:36:16.473628 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:36:16.475938 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84555a4d-bed8-4bcc-9af3-350ae4de2f54 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:16.485732 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84555a4d-bed8-4bcc-9af3-350ae4de2f54 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:16.490638 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032b9df8-7b3a-4a4d-bd8b-9db03597e687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:16.491348 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032b9df8-7b3a-4a4d-bd8b-9db03597e687 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:16.491630 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032b9df8-7b3a-4a4d-bd8b-9db03597e687 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:16.491906 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032b9df8-7b3a-4a4d-bd8b-9db03597e687 admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:16.493181 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032b9df8-7b3a-4a4d-bd8b-9db03597e687 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:16.495127 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032b9df8-7b3a-4a4d-bd8b-9db03597e687 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:16.610604 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1078/2125] 10.222.0.66 () {60 vars in 1152 bytes} [Mon Jul 18 02:36:16 2022] POST /identity/v3/projects => generated 375 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:36:16.611707 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df66b887-6508-4fa2-bb0b-2ea745876b3c admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:16.622740 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df66b887-6508-4fa2-bb0b-2ea745876b3c admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:16.682793 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2b5bab-1ece-463d-a15d-7ee49a48ceae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:16.683446 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2b5bab-1ece-463d-a15d-7ee49a48ceae admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:16.683736 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2b5bab-1ece-463d-a15d-7ee49a48ceae admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:16.684040 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2b5bab-1ece-463d-a15d-7ee49a48ceae admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:16.684741 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2b5bab-1ece-463d-a15d-7ee49a48ceae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:16.686697 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2b5bab-1ece-463d-a15d-7ee49a48ceae admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:16.793437 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1048/2126] 10.222.0.66 () {60 vars in 1143 bytes} [Mon Jul 18 02:36:16 2022] POST /identity/v3/users => generated 288 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:36:16.797269 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032b9df8-7b3a-4a4d-bd8b-9db03597e687 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:16.807324 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032b9df8-7b3a-4a4d-bd8b-9db03597e687 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:16.810320 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:36:16.814794 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0762c4f4-ff57-40b0-8fe8-57f97912ac51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:16.821193 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0762c4f4-ff57-40b0-8fe8-57f97912ac51 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:16.821193 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0762c4f4-ff57-40b0-8fe8-57f97912ac51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:16.821193 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0762c4f4-ff57-40b0-8fe8-57f97912ac51 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:16.821193 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0762c4f4-ff57-40b0-8fe8-57f97912ac51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:16.821193 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0762c4f4-ff57-40b0-8fe8-57f97912ac51 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:16.829986 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1079/2127] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:36:16 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:36:16.833888 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2b5bab-1ece-463d-a15d-7ee49a48ceae admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:16.844720 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2b5bab-1ece-463d-a15d-7ee49a48ceae admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:16.849702 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce38a35-c56c-48c4-ad6b-e280b5ef3a81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:16.850340 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce38a35-c56c-48c4-ad6b-e280b5ef3a81 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:16.850631 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce38a35-c56c-48c4-ad6b-e280b5ef3a81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:16.850932 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce38a35-c56c-48c4-ad6b-e280b5ef3a81 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:16.852256 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce38a35-c56c-48c4-ad6b-e280b5ef3a81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:16.854654 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce38a35-c56c-48c4-ad6b-e280b5ef3a81 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:16.865154 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1049/2128] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:36:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:36:16.869747 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0762c4f4-ff57-40b0-8fe8-57f97912ac51 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:16.880124 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0762c4f4-ff57-40b0-8fe8-57f97912ac51 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:16.885484 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99472a8a-e5b9-4a0d-8270-e80f842c9b7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:16.890996 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99472a8a-e5b9-4a0d-8270-e80f842c9b7a admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:16.890996 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99472a8a-e5b9-4a0d-8270-e80f842c9b7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:16.890996 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99472a8a-e5b9-4a0d-8270-e80f842c9b7a admin admin] PATH_INFO: `/v3/projects/32003435719747cfa3cf041b976d7d50/users/67c5ddc3cf8944a8808ec16092df4a41/roles/65feabf3b27944a8b01a86c8ac608457` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:16.905636 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99472a8a-e5b9-4a0d-8270-e80f842c9b7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=32003435719747cfa3cf041b976d7d50, user_id=67c5ddc3cf8944a8808ec16092df4a41, role_id=65feabf3b27944a8b01a86c8ac608457)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:16.908561 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99472a8a-e5b9-4a0d-8270-e80f842c9b7a admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:17.030951 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1080/2129] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:36:16 2022] PUT /identity/v3/projects/32003435719747cfa3cf041b976d7d50/users/67c5ddc3cf8944a8808ec16092df4a41/roles/65feabf3b27944a8b01a86c8ac608457 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:36:17.041364 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3528ee92-814f-425e-bea6-9efc3429e164 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:17.041716 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3528ee92-814f-425e-bea6-9efc3429e164 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:17.042023 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3528ee92-814f-425e-bea6-9efc3429e164 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:17.073611 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-3528ee92-814f-425e-bea6-9efc3429e164 None None] MFA Rules not processed for user `67c5ddc3cf8944a8808ec16092df4a41`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:36:17.075113 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-3528ee92-814f-425e-bea6-9efc3429e164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:17.166786 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:36:17.180096 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99472a8a-e5b9-4a0d-8270-e80f842c9b7a admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:17.185679 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-99472a8a-e5b9-4a0d-8270-e80f842c9b7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:17.197680 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1050/2130] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:36:17 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:36:17.208405 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3528ee92-814f-425e-bea6-9efc3429e164 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:17.209839 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-3528ee92-814f-425e-bea6-9efc3429e164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:17.258470 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99472a8a-e5b9-4a0d-8270-e80f842c9b7a admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:17.263703 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4229c26-a058-4ac3-9e19-ec3225aa5034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:17.264760 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4229c26-a058-4ac3-9e19-ec3225aa5034 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:17.265045 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4229c26-a058-4ac3-9e19-ec3225aa5034 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:17.265322 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4229c26-a058-4ac3-9e19-ec3225aa5034 admin admin] PATH_INFO: `/v3/users/9c93dbc940cd4deea0fcb99d0ca0c542` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:17.267212 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4229c26-a058-4ac3-9e19-ec3225aa5034 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c93dbc940cd4deea0fcb99d0ca0c542)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:17.269698 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4229c26-a058-4ac3-9e19-ec3225aa5034 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:17.270185 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3528ee92-814f-425e-bea6-9efc3429e164 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:17.274442 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c106600b-6955-4f77-8316-ff28e17dd1fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd49388facdf446885e55b45ef91f01e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:17.275171 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c106600b-6955-4f77-8316-ff28e17dd1fb service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:17.275459 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c106600b-6955-4f77-8316-ff28e17dd1fb service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:17.275787 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c106600b-6955-4f77-8316-ff28e17dd1fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:17.284754 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c106600b-6955-4f77-8316-ff28e17dd1fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:17.286296 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c106600b-6955-4f77-8316-ff28e17dd1fb service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:17.296199 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1051/2131] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:36:17 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 02:36:17.451562 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-c4229c26-a058-4ac3-9e19-ec3225aa5034 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c93dbc940cd4deea0fcb99d0ca0c542'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:17.499434 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c106600b-6955-4f77-8316-ff28e17dd1fb service cinder] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:17.503407 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-c106600b-6955-4f77-8316-ff28e17dd1fb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:17.512859 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c106600b-6955-4f77-8316-ff28e17dd1fb service cinder] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:17.516870 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22df484-670f-4676-8637-7a71619121a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd49388facdf446885e55b45ef91f01e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:17.517513 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22df484-670f-4676-8637-7a71619121a4 service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:17.517802 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22df484-670f-4676-8637-7a71619121a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:17.518415 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22df484-670f-4676-8637-7a71619121a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:17.520384 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-f22df484-670f-4676-8637-7a71619121a4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:17.539339 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:36:17.540543 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-c4229c26-a058-4ac3-9e19-ec3225aa5034 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c93dbc940cd4deea0fcb99d0ca0c542'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:17.548477 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-c4229c26-a058-4ac3-9e19-ec3225aa5034 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c93dbc940cd4deea0fcb99d0ca0c542'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:17.549204 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-c4229c26-a058-4ac3-9e19-ec3225aa5034 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c93dbc940cd4deea0fcb99d0ca0c542'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:17.562152 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-c4229c26-a058-4ac3-9e19-ec3225aa5034 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c93dbc940cd4deea0fcb99d0ca0c542'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:17.568424 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1081/2132] 10.222.0.66 () {58 vars in 1223 bytes} [Mon Jul 18 02:36:17 2022] DELETE /identity/v3/users/9c93dbc940cd4deea0fcb99d0ca0c542 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:36:17.573665 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709634cd-31ab-44e7-b272-229e48c667a2 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:17.573957 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709634cd-31ab-44e7-b272-229e48c667a2 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:17.574267 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709634cd-31ab-44e7-b272-229e48c667a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:17.585498 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22df484-670f-4676-8637-7a71619121a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:17.586982 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22df484-670f-4676-8637-7a71619121a4 service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:17.591195 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-709634cd-31ab-44e7-b272-229e48c667a2 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:36:17.592384 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-709634cd-31ab-44e7-b272-229e48c667a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:17.596606 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1052/2133] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:36:17 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:36:17.664026 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1082/2134] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:36:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 02:36:17.668961 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22df484-670f-4676-8637-7a71619121a4 service cinder] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:17.670411 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-f22df484-670f-4676-8637-7a71619121a4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:17.726980 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22df484-670f-4676-8637-7a71619121a4 service cinder] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:17.781083 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a78a5d-b641-4d7f-9934-f6f5e5e336a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eaf19a259c2741cea375a38763e33cdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:17.781737 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a78a5d-b641-4d7f-9934-f6f5e5e336a5 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:17.782018 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a78a5d-b641-4d7f-9934-f6f5e5e336a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:17.782322 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a78a5d-b641-4d7f-9934-f6f5e5e336a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:17.791891 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a78a5d-b641-4d7f-9934-f6f5e5e336a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:17.793331 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a78a5d-b641-4d7f-9934-f6f5e5e336a5 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:17.803992 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1053/2135] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:36:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:36:17.878321 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:INSERT=5,keystone:DELETE=1,keystone:SELECT=94 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:36:18.192448 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:36:18.265709 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709634cd-31ab-44e7-b272-229e48c667a2 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:18.267793 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-709634cd-31ab-44e7-b272-229e48c667a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:18.280363 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709634cd-31ab-44e7-b272-229e48c667a2 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:18.285018 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc212b9e-e0ad-456c-85a9-2a0b0d36abd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:18.285891 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc212b9e-e0ad-456c-85a9-2a0b0d36abd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:18.286200 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc212b9e-e0ad-456c-85a9-2a0b0d36abd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:18.298702 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc212b9e-e0ad-456c-85a9-2a0b0d36abd8 admin admin] PATH_INFO: `/v3/projects/ab7cf9a77e614266a1c0a908babb8515` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:18.298702 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc212b9e-e0ad-456c-85a9-2a0b0d36abd8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab7cf9a77e614266a1c0a908babb8515)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:18.298702 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc212b9e-e0ad-456c-85a9-2a0b0d36abd8 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:18.556869 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-bc212b9e-e0ad-456c-85a9-2a0b0d36abd8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab7cf9a77e614266a1c0a908babb8515', 'request_id': 'req-bc212b9e-e0ad-456c-85a9-2a0b0d36abd8'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:18.562937 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-bc212b9e-e0ad-456c-85a9-2a0b0d36abd8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab7cf9a77e614266a1c0a908babb8515', 'request_id': 'req-bc212b9e-e0ad-456c-85a9-2a0b0d36abd8'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:18.633179 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:36:18.666575 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-bc212b9e-e0ad-456c-85a9-2a0b0d36abd8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab7cf9a77e614266a1c0a908babb8515', 'request_id': 'req-bc212b9e-e0ad-456c-85a9-2a0b0d36abd8'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:18.678800 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-bc212b9e-e0ad-456c-85a9-2a0b0d36abd8 admin admin] The token cache is being invalidate because project ab7cf9a77e614266a1c0a908babb8515 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 02:36:18.679081 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-bc212b9e-e0ad-456c-85a9-2a0b0d36abd8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:18.679774 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-bc212b9e-e0ad-456c-85a9-2a0b0d36abd8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:18.681336 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1083/2136] 10.222.0.66 () {58 vars in 1232 bytes} [Mon Jul 18 02:36:18 2022] DELETE /identity/v3/projects/ab7cf9a77e614266a1c0a908babb8515 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:36:18.686354 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b43f7c-967e-4909-84a4-997c706f046e None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:18.686657 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b43f7c-967e-4909-84a4-997c706f046e None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:18.686926 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b43f7c-967e-4909-84a4-997c706f046e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:18.712327 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-e8b43f7c-967e-4909-84a4-997c706f046e None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:36:18.713542 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-e8b43f7c-967e-4909-84a4-997c706f046e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:18.756058 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:36:18.772661 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1054/2137] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:36:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:36:18.781040 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af051efb-2a91-4891-9e9d-02312b34255a None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:18.781428 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af051efb-2a91-4891-9e9d-02312b34255a None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:18.781770 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af051efb-2a91-4891-9e9d-02312b34255a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:18.800279 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-af051efb-2a91-4891-9e9d-02312b34255a None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:36:18.801519 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-af051efb-2a91-4891-9e9d-02312b34255a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:18.813134 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1084/2138] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:36:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:36:18.814279 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b43f7c-967e-4909-84a4-997c706f046e None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:18.824528 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b43f7c-967e-4909-84a4-997c706f046e None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:18.829222 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431c542e-16ff-4677-a598-ffff77bf2064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:18.829880 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431c542e-16ff-4677-a598-ffff77bf2064 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:18.830194 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431c542e-16ff-4677-a598-ffff77bf2064 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:18.830474 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431c542e-16ff-4677-a598-ffff77bf2064 admin admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:18.831203 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431c542e-16ff-4677-a598-ffff77bf2064 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:36:18.831513 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431c542e-16ff-4677-a598-ffff77bf2064 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:18.833138 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431c542e-16ff-4677-a598-ffff77bf2064 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:18.845181 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1055/2139] 10.222.0.66 () {58 vars in 1152 bytes} [Mon Jul 18 02:36:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:36:18.849815 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af051efb-2a91-4891-9e9d-02312b34255a None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:18.860547 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af051efb-2a91-4891-9e9d-02312b34255a None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:18.866044 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028db972-867f-41b0-9381-0681383fe9ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:18.867909 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028db972-867f-41b0-9381-0681383fe9ff admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:18.868696 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028db972-867f-41b0-9381-0681383fe9ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:18.869157 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028db972-867f-41b0-9381-0681383fe9ff admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:18.870108 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028db972-867f-41b0-9381-0681383fe9ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:18.874735 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028db972-867f-41b0-9381-0681383fe9ff admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:18.964413 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1085/2140] 10.222.0.66 () {60 vars in 1152 bytes} [Mon Jul 18 02:36:18 2022] POST /identity/v3/projects => generated 375 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:36:18.966186 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431c542e-16ff-4677-a598-ffff77bf2064 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:18.976927 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431c542e-16ff-4677-a598-ffff77bf2064 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:19.026682 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:36:19.053722 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4864a58f-1c89-44f4-831c-fdd722230e2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:19.055157 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4864a58f-1c89-44f4-831c-fdd722230e2d admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:19.056898 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4864a58f-1c89-44f4-831c-fdd722230e2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:19.056898 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4864a58f-1c89-44f4-831c-fdd722230e2d admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:19.056898 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4864a58f-1c89-44f4-831c-fdd722230e2d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:19.058679 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4864a58f-1c89-44f4-831c-fdd722230e2d admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:19.140009 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1056/2141] 10.222.0.66 () {60 vars in 1143 bytes} [Mon Jul 18 02:36:18 2022] POST /identity/v3/users => generated 288 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:36:19.143993 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028db972-867f-41b0-9381-0681383fe9ff admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:19.154836 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028db972-867f-41b0-9381-0681383fe9ff admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:19.160431 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c98a6c-f437-4d0f-b389-d210d604a45b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:19.161208 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c98a6c-f437-4d0f-b389-d210d604a45b admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:19.168214 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c98a6c-f437-4d0f-b389-d210d604a45b admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:19.168214 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c98a6c-f437-4d0f-b389-d210d604a45b admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:19.168214 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c98a6c-f437-4d0f-b389-d210d604a45b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:19.168214 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c98a6c-f437-4d0f-b389-d210d604a45b admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:19.175185 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1086/2142] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:36:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:36:19.179972 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4864a58f-1c89-44f4-831c-fdd722230e2d admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:19.193323 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4864a58f-1c89-44f4-831c-fdd722230e2d admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:19.199684 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e436add1-51a5-43be-bea4-bc319f828449 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:19.200372 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e436add1-51a5-43be-bea4-bc319f828449 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:19.200664 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e436add1-51a5-43be-bea4-bc319f828449 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:19.200945 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e436add1-51a5-43be-bea4-bc319f828449 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:19.201769 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e436add1-51a5-43be-bea4-bc319f828449 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:19.203431 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e436add1-51a5-43be-bea4-bc319f828449 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:19.217978 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1057/2143] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:36:19 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:36:19.218995 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c98a6c-f437-4d0f-b389-d210d604a45b admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:19.234766 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c98a6c-f437-4d0f-b389-d210d604a45b admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:19.240616 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e5ca7a-e4f9-4f47-8d10-05e317225f69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:19.242164 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e5ca7a-e4f9-4f47-8d10-05e317225f69 admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:19.242705 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e5ca7a-e4f9-4f47-8d10-05e317225f69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:19.243030 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e5ca7a-e4f9-4f47-8d10-05e317225f69 admin admin] PATH_INFO: `/v3/projects/5d84de9074bb4d9a87d20326924df7e6/users/2b858f9e4b474eaa918623a9148f28b8/roles/65feabf3b27944a8b01a86c8ac608457` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:19.268540 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e5ca7a-e4f9-4f47-8d10-05e317225f69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d84de9074bb4d9a87d20326924df7e6, user_id=2b858f9e4b474eaa918623a9148f28b8, role_id=65feabf3b27944a8b01a86c8ac608457)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:19.271702 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e5ca7a-e4f9-4f47-8d10-05e317225f69 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:19.372061 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1087/2144] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:36:19 2022] PUT /identity/v3/projects/5d84de9074bb4d9a87d20326924df7e6/users/2b858f9e4b474eaa918623a9148f28b8/roles/65feabf3b27944a8b01a86c8ac608457 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:36:19.382951 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4f5aab-7344-44e9-9a42-b62e5fae3149 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:19.383451 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4f5aab-7344-44e9-9a42-b62e5fae3149 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:19.383746 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4f5aab-7344-44e9-9a42-b62e5fae3149 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:19.414438 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-0a4f5aab-7344-44e9-9a42-b62e5fae3149 None None] MFA Rules not processed for user `2b858f9e4b474eaa918623a9148f28b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:36:19.416557 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-0a4f5aab-7344-44e9-9a42-b62e5fae3149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:19.546137 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1058/2145] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:36:19 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:36:19.573112 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e5ca7a-e4f9-4f47-8d10-05e317225f69 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:19.575762 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-e9e5ca7a-e4f9-4f47-8d10-05e317225f69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:19.649259 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e5ca7a-e4f9-4f47-8d10-05e317225f69 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:19.649259 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad28582-a1e5-4e54-8caa-0f4d1d3274f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd49388facdf446885e55b45ef91f01e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:19.649981 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad28582-a1e5-4e54-8caa-0f4d1d3274f6 service cinder] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:19.650218 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad28582-a1e5-4e54-8caa-0f4d1d3274f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:19.650693 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad28582-a1e5-4e54-8caa-0f4d1d3274f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:19.660363 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad28582-a1e5-4e54-8caa-0f4d1d3274f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:19.661960 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad28582-a1e5-4e54-8caa-0f4d1d3274f6 service cinder] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:19.675265 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:36:19.680671 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1088/2146] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:36:19 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:36:19.941153 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:36:20.507205 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:36:21.093180 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=83,keystone:INSERT=3,keystone:DELETE=14 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:36:21.615196 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:36:21.911506 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4f5aab-7344-44e9-9a42-b62e5fae3149 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:21.913780 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-0a4f5aab-7344-44e9-9a42-b62e5fae3149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:21.988335 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4f5aab-7344-44e9-9a42-b62e5fae3149 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:21.993752 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72701875-80b2-49b9-954b-2ad7f1aa6225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:21.998107 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72701875-80b2-49b9-954b-2ad7f1aa6225 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:21.998107 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72701875-80b2-49b9-954b-2ad7f1aa6225 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:21.998107 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72701875-80b2-49b9-954b-2ad7f1aa6225 admin admin] PATH_INFO: `/v3/users/67c5ddc3cf8944a8808ec16092df4a41` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:21.998107 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72701875-80b2-49b9-954b-2ad7f1aa6225 admin admin] RBAC: Authorizing `identity:delete_user(user_id=67c5ddc3cf8944a8808ec16092df4a41)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:21.999406 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72701875-80b2-49b9-954b-2ad7f1aa6225 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:22.233982 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-72701875-80b2-49b9-954b-2ad7f1aa6225 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '67c5ddc3cf8944a8808ec16092df4a41'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:22.357783 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-72701875-80b2-49b9-954b-2ad7f1aa6225 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '67c5ddc3cf8944a8808ec16092df4a41'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:22.358565 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-72701875-80b2-49b9-954b-2ad7f1aa6225 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '67c5ddc3cf8944a8808ec16092df4a41'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:22.371878 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-72701875-80b2-49b9-954b-2ad7f1aa6225 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '67c5ddc3cf8944a8808ec16092df4a41'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:22.372672 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-72701875-80b2-49b9-954b-2ad7f1aa6225 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '67c5ddc3cf8944a8808ec16092df4a41'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:22.405329 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1059/2147] 10.222.0.66 () {58 vars in 1223 bytes} [Mon Jul 18 02:36:21 2022] DELETE /identity/v3/users/67c5ddc3cf8944a8808ec16092df4a41 => generated 0 bytes in 492 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:36:22.407381 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cff99b1-bac5-4975-91b7-84614b3466b2 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:22.407777 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cff99b1-bac5-4975-91b7-84614b3466b2 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:22.408180 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cff99b1-bac5-4975-91b7-84614b3466b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:22.431861 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-7cff99b1-bac5-4975-91b7-84614b3466b2 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:36:22.433463 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-7cff99b1-bac5-4975-91b7-84614b3466b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:22.488608 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72701875-80b2-49b9-954b-2ad7f1aa6225 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:22.490154 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-72701875-80b2-49b9-954b-2ad7f1aa6225 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:22.520649 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1089/2148] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:36:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:36:22.532067 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cff99b1-bac5-4975-91b7-84614b3466b2 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:22.533489 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-7cff99b1-bac5-4975-91b7-84614b3466b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:22.573486 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72701875-80b2-49b9-954b-2ad7f1aa6225 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:22.575929 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9efcee-6b23-4fc3-bf80-95fedc617299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:22.577120 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9efcee-6b23-4fc3-bf80-95fedc617299 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:22.577264 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9efcee-6b23-4fc3-bf80-95fedc617299 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:22.577720 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9efcee-6b23-4fc3-bf80-95fedc617299 admin admin] PATH_INFO: `/v3/users/973f5fea84de4c9ea4b806a825f5c533` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:22.579438 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9efcee-6b23-4fc3-bf80-95fedc617299 admin admin] RBAC: Authorizing `identity:delete_user(user_id=973f5fea84de4c9ea4b806a825f5c533)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:22.581797 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9efcee-6b23-4fc3-bf80-95fedc617299 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:22.606566 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cff99b1-bac5-4975-91b7-84614b3466b2 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:22.609430 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2672676c-ed18-4a0d-b38b-df503244a835 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eaf19a259c2741cea375a38763e33cdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:22.610356 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2672676c-ed18-4a0d-b38b-df503244a835 service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:22.610809 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2672676c-ed18-4a0d-b38b-df503244a835 service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:22.611302 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2672676c-ed18-4a0d-b38b-df503244a835 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:22.624542 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2672676c-ed18-4a0d-b38b-df503244a835 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:22.628489 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2672676c-ed18-4a0d-b38b-df503244a835 service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:22.638821 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1090/2149] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:36:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:36:22.653773 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:DELETE=3,keystone:SELECT=97 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:36:22.816611 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2672676c-ed18-4a0d-b38b-df503244a835 service neutron] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:22.818063 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-2672676c-ed18-4a0d-b38b-df503244a835 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:22.832877 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2672676c-ed18-4a0d-b38b-df503244a835 service neutron] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:22.841132 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d3ea8c-45a7-4578-af2b-4663f16b01bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:22.842034 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d3ea8c-45a7-4578-af2b-4663f16b01bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:22.842352 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d3ea8c-45a7-4578-af2b-4663f16b01bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:22.842632 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d3ea8c-45a7-4578-af2b-4663f16b01bc admin admin] PATH_INFO: `/v3/users/39f6e1501362435d8485ac5eb4b9f315` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:22.843847 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d3ea8c-45a7-4578-af2b-4663f16b01bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=39f6e1501362435d8485ac5eb4b9f315)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:22.845902 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d3ea8c-45a7-4578-af2b-4663f16b01bc admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:22.875500 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-1c9efcee-6b23-4fc3-bf80-95fedc617299 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '973f5fea84de4c9ea4b806a825f5c533'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:22.991464 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-1c9efcee-6b23-4fc3-bf80-95fedc617299 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '973f5fea84de4c9ea4b806a825f5c533'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:22.994504 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-1c9efcee-6b23-4fc3-bf80-95fedc617299 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '973f5fea84de4c9ea4b806a825f5c533'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:23.008093 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-1c9efcee-6b23-4fc3-bf80-95fedc617299 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '973f5fea84de4c9ea4b806a825f5c533'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:23.008881 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-1c9efcee-6b23-4fc3-bf80-95fedc617299 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '973f5fea84de4c9ea4b806a825f5c533'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:23.021282 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1060/2150] 10.222.0.66 () {58 vars in 1223 bytes} [Mon Jul 18 02:36:22 2022] DELETE /identity/v3/users/973f5fea84de4c9ea4b806a825f5c533 => generated 0 bytes in 533 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 02:36:23.026283 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7467cc20-f424-4020-9550-74e11c07ec2e None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:23.026572 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7467cc20-f424-4020-9550-74e11c07ec2e None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:23.026849 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7467cc20-f424-4020-9550-74e11c07ec2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:23.044827 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-7467cc20-f424-4020-9550-74e11c07ec2e None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:36:23.046136 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-7467cc20-f424-4020-9550-74e11c07ec2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:23.110249 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1061/2151] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:36:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:36:23.114300 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-c6d3ea8c-45a7-4578-af2b-4663f16b01bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39f6e1501362435d8485ac5eb4b9f315'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:23.117759 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7467cc20-f424-4020-9550-74e11c07ec2e None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:23.119276 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-7467cc20-f424-4020-9550-74e11c07ec2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:23.179277 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7467cc20-f424-4020-9550-74e11c07ec2e None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:23.184158 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d345e47e-af64-453f-b471-1bea83602fe5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eaf19a259c2741cea375a38763e33cdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:23.184949 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d345e47e-af64-453f-b471-1bea83602fe5 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:23.185331 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d345e47e-af64-453f-b471-1bea83602fe5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:23.185636 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-c6d3ea8c-45a7-4578-af2b-4663f16b01bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39f6e1501362435d8485ac5eb4b9f315'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:23.185937 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d345e47e-af64-453f-b471-1bea83602fe5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:23.193778 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-c6d3ea8c-45a7-4578-af2b-4663f16b01bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39f6e1501362435d8485ac5eb4b9f315'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:23.195161 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-c6d3ea8c-45a7-4578-af2b-4663f16b01bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39f6e1501362435d8485ac5eb4b9f315'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:23.197460 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d345e47e-af64-453f-b471-1bea83602fe5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:23.199003 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d345e47e-af64-453f-b471-1bea83602fe5 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:23.209249 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1062/2152] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:36:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 02:36:23.210158 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d345e47e-af64-453f-b471-1bea83602fe5 service neutron] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:23.212009 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-d345e47e-af64-453f-b471-1bea83602fe5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:23.214064 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-c6d3ea8c-45a7-4578-af2b-4663f16b01bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39f6e1501362435d8485ac5eb4b9f315'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:23.223737 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1091/2153] 10.222.0.66 () {58 vars in 1223 bytes} [Mon Jul 18 02:36:22 2022] DELETE /identity/v3/users/39f6e1501362435d8485ac5eb4b9f315 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 02:36:23.229697 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d345e47e-af64-453f-b471-1bea83602fe5 service neutron] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:23.295042 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc5f6f9-505f-4d84-833f-135bdc6d541b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:23.296688 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc5f6f9-505f-4d84-833f-135bdc6d541b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:23.296976 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc5f6f9-505f-4d84-833f-135bdc6d541b admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:23.297258 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc5f6f9-505f-4d84-833f-135bdc6d541b admin admin] PATH_INFO: `/v3/projects/32003435719747cfa3cf041b976d7d50` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:23.298517 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc5f6f9-505f-4d84-833f-135bdc6d541b admin admin] RBAC: Authorizing `identity:delete_project(project_id=32003435719747cfa3cf041b976d7d50)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:23.300497 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc5f6f9-505f-4d84-833f-135bdc6d541b admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:23.432734 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-edc5f6f9-505f-4d84-833f-135bdc6d541b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32003435719747cfa3cf041b976d7d50', 'request_id': 'req-edc5f6f9-505f-4d84-833f-135bdc6d541b'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:23.438178 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-edc5f6f9-505f-4d84-833f-135bdc6d541b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32003435719747cfa3cf041b976d7d50', 'request_id': 'req-edc5f6f9-505f-4d84-833f-135bdc6d541b'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:23.444444 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-edc5f6f9-505f-4d84-833f-135bdc6d541b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32003435719747cfa3cf041b976d7d50', 'request_id': 'req-edc5f6f9-505f-4d84-833f-135bdc6d541b'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:23.513745 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d3ea8c-45a7-4578-af2b-4663f16b01bc admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:23.515857 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-c6d3ea8c-45a7-4578-af2b-4663f16b01bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:23.529869 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d3ea8c-45a7-4578-af2b-4663f16b01bc admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:23.549261 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-edc5f6f9-505f-4d84-833f-135bdc6d541b admin admin] The token cache is being invalidate because project 32003435719747cfa3cf041b976d7d50 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 02:36:23.556530 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-edc5f6f9-505f-4d84-833f-135bdc6d541b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:23.556530 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-edc5f6f9-505f-4d84-833f-135bdc6d541b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:23.557918 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1063/2154] 10.222.0.66 () {58 vars in 1232 bytes} [Mon Jul 18 02:36:23 2022] DELETE /identity/v3/projects/32003435719747cfa3cf041b976d7d50 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:36:23.642922 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f12a878-593e-4db7-9db3-4f30decb1bce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:23.644062 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f12a878-593e-4db7-9db3-4f30decb1bce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:23.644342 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f12a878-593e-4db7-9db3-4f30decb1bce admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:23.644613 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f12a878-593e-4db7-9db3-4f30decb1bce admin admin] PATH_INFO: `/v3/projects/d0695b3617d845118c00ba7cff6d312a` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:23.645818 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f12a878-593e-4db7-9db3-4f30decb1bce admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0695b3617d845118c00ba7cff6d312a)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:23.648334 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f12a878-593e-4db7-9db3-4f30decb1bce admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:23.697173 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc5f6f9-505f-4d84-833f-135bdc6d541b admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:23.699019 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-edc5f6f9-505f-4d84-833f-135bdc6d541b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:23.716075 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc5f6f9-505f-4d84-833f-135bdc6d541b admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:23.721578 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127fd8e8-ac23-407d-aada-d7ce5e9970c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:23.723123 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127fd8e8-ac23-407d-aada-d7ce5e9970c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:23.723599 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127fd8e8-ac23-407d-aada-d7ce5e9970c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:23.724153 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127fd8e8-ac23-407d-aada-d7ce5e9970c2 admin admin] PATH_INFO: `/v3/projects/f30256b4abd24f37a048eb5053039845` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:23.725544 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127fd8e8-ac23-407d-aada-d7ce5e9970c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f30256b4abd24f37a048eb5053039845)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:23.728302 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127fd8e8-ac23-407d-aada-d7ce5e9970c2 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:23.806971 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-2f12a878-593e-4db7-9db3-4f30decb1bce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0695b3617d845118c00ba7cff6d312a', 'request_id': 'req-2f12a878-593e-4db7-9db3-4f30decb1bce'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:23.812179 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-2f12a878-593e-4db7-9db3-4f30decb1bce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0695b3617d845118c00ba7cff6d312a', 'request_id': 'req-2f12a878-593e-4db7-9db3-4f30decb1bce'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:23.818237 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:36:23.944231 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-127fd8e8-ac23-407d-aada-d7ce5e9970c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f30256b4abd24f37a048eb5053039845', 'request_id': 'req-127fd8e8-ac23-407d-aada-d7ce5e9970c2'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:23.950399 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-127fd8e8-ac23-407d-aada-d7ce5e9970c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f30256b4abd24f37a048eb5053039845', 'request_id': 'req-127fd8e8-ac23-407d-aada-d7ce5e9970c2'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:23.951854 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-2f12a878-593e-4db7-9db3-4f30decb1bce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0695b3617d845118c00ba7cff6d312a', 'request_id': 'req-2f12a878-593e-4db7-9db3-4f30decb1bce'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:23.957288 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-127fd8e8-ac23-407d-aada-d7ce5e9970c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f30256b4abd24f37a048eb5053039845', 'request_id': 'req-127fd8e8-ac23-407d-aada-d7ce5e9970c2'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:23.964194 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-2f12a878-593e-4db7-9db3-4f30decb1bce admin admin] The token cache is being invalidate because project d0695b3617d845118c00ba7cff6d312a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 02:36:23.964491 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-2f12a878-593e-4db7-9db3-4f30decb1bce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:23.965147 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-2f12a878-593e-4db7-9db3-4f30decb1bce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:23.966794 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1092/2155] 10.222.0.66 () {58 vars in 1232 bytes} [Mon Jul 18 02:36:23 2022] DELETE /identity/v3/projects/d0695b3617d845118c00ba7cff6d312a => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:36:23.971905 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e81a80-56e6-4e32-ac5c-23920d642fb8 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:23.972196 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e81a80-56e6-4e32-ac5c-23920d642fb8 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:23.972485 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e81a80-56e6-4e32-ac5c-23920d642fb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:23.992233 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-45e81a80-56e6-4e32-ac5c-23920d642fb8 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:36:23.993625 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-45e81a80-56e6-4e32-ac5c-23920d642fb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:24.053867 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1093/2156] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:36:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 02:36:24.054062 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-127fd8e8-ac23-407d-aada-d7ce5e9970c2 admin admin] The token cache is being invalidate because project f30256b4abd24f37a048eb5053039845 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 02:36:24.054193 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-127fd8e8-ac23-407d-aada-d7ce5e9970c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:24.054827 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-127fd8e8-ac23-407d-aada-d7ce5e9970c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:24.056382 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1064/2157] 10.222.0.66 () {58 vars in 1232 bytes} [Mon Jul 18 02:36:23 2022] DELETE /identity/v3/projects/f30256b4abd24f37a048eb5053039845 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:36:24.061581 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e81a80-56e6-4e32-ac5c-23920d642fb8 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:24.063039 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-45e81a80-56e6-4e32-ac5c-23920d642fb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:24.063319 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8127cb33-fbb8-4ce5-85e4-6d8e8fa1db11 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:24.063610 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8127cb33-fbb8-4ce5-85e4-6d8e8fa1db11 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:24.063891 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8127cb33-fbb8-4ce5-85e4-6d8e8fa1db11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:24.075929 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e81a80-56e6-4e32-ac5c-23920d642fb8 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:24.081700 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636f93c0-e0f6-499c-883f-e7935625a944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:24.082630 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636f93c0-e0f6-499c-883f-e7935625a944 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:24.082962 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636f93c0-e0f6-499c-883f-e7935625a944 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:24.083240 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636f93c0-e0f6-499c-883f-e7935625a944 admin admin] PATH_INFO: `/v3/users/ae77aa9557b1433ea7681caffd85b634` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:24.084473 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636f93c0-e0f6-499c-883f-e7935625a944 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae77aa9557b1433ea7681caffd85b634)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:24.086676 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-8127cb33-fbb8-4ce5-85e4-6d8e8fa1db11 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:36:24.087237 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636f93c0-e0f6-499c-883f-e7935625a944 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:24.087994 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-8127cb33-fbb8-4ce5-85e4-6d8e8fa1db11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:24.096569 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1065/2158] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:36:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:36:24.101529 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8127cb33-fbb8-4ce5-85e4-6d8e8fa1db11 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:24.110800 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8127cb33-fbb8-4ce5-85e4-6d8e8fa1db11 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:24.115269 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32491f73-4498-47eb-992c-5be4a396f462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:24.115962 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32491f73-4498-47eb-992c-5be4a396f462 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:24.116254 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32491f73-4498-47eb-992c-5be4a396f462 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:24.116520 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32491f73-4498-47eb-992c-5be4a396f462 admin admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:24.117304 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32491f73-4498-47eb-992c-5be4a396f462 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:36:24.117541 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32491f73-4498-47eb-992c-5be4a396f462 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:24.119202 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32491f73-4498-47eb-992c-5be4a396f462 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:24.130930 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1066/2159] 10.222.0.66 () {58 vars in 1152 bytes} [Mon Jul 18 02:36:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:36:24.135821 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32491f73-4498-47eb-992c-5be4a396f462 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:24.145583 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32491f73-4498-47eb-992c-5be4a396f462 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:24.150863 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cf08cf-3d80-441e-a396-c08b83012ef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:24.151491 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cf08cf-3d80-441e-a396-c08b83012ef7 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:24.151773 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cf08cf-3d80-441e-a396-c08b83012ef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:24.152050 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cf08cf-3d80-441e-a396-c08b83012ef7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:24.152723 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cf08cf-3d80-441e-a396-c08b83012ef7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:24.154682 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cf08cf-3d80-441e-a396-c08b83012ef7 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:24.209376 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:36:24.261846 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1067/2160] 10.222.0.66 () {60 vars in 1152 bytes} [Mon Jul 18 02:36:24 2022] POST /identity/v3/projects => generated 391 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 02:36:24.266410 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cf08cf-3d80-441e-a396-c08b83012ef7 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:24.291683 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-636f93c0-e0f6-499c-883f-e7935625a944 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae77aa9557b1433ea7681caffd85b634'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:24.332825 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cf08cf-3d80-441e-a396-c08b83012ef7 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:24.338229 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9bca8d-f193-4935-81e6-a46e2b41a438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:24.338900 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9bca8d-f193-4935-81e6-a46e2b41a438 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:24.339185 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9bca8d-f193-4935-81e6-a46e2b41a438 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:24.339461 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9bca8d-f193-4935-81e6-a46e2b41a438 admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:24.340195 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9bca8d-f193-4935-81e6-a46e2b41a438 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:24.342150 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9bca8d-f193-4935-81e6-a46e2b41a438 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:24.374424 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-636f93c0-e0f6-499c-883f-e7935625a944 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae77aa9557b1433ea7681caffd85b634'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:24.382695 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-636f93c0-e0f6-499c-883f-e7935625a944 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae77aa9557b1433ea7681caffd85b634'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:24.385801 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-636f93c0-e0f6-499c-883f-e7935625a944 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae77aa9557b1433ea7681caffd85b634'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:24.395880 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-636f93c0-e0f6-499c-883f-e7935625a944 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae77aa9557b1433ea7681caffd85b634'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:24.402508 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1094/2161] 10.222.0.66 () {58 vars in 1223 bytes} [Mon Jul 18 02:36:24 2022] DELETE /identity/v3/users/ae77aa9557b1433ea7681caffd85b634 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 02:36:24.446861 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1068/2162] 10.222.0.66 () {60 vars in 1143 bytes} [Mon Jul 18 02:36:24 2022] POST /identity/v3/users => generated 296 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 02:36:24.448207 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636f93c0-e0f6-499c-883f-e7935625a944 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:24.449749 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-636f93c0-e0f6-499c-883f-e7935625a944 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:24.520794 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636f93c0-e0f6-499c-883f-e7935625a944 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:24.526604 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f66dfeb-7366-45b6-8adb-cb588abefe57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:24.527312 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f66dfeb-7366-45b6-8adb-cb588abefe57 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:24.527819 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f66dfeb-7366-45b6-8adb-cb588abefe57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:24.528580 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f66dfeb-7366-45b6-8adb-cb588abefe57 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:24.529423 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f66dfeb-7366-45b6-8adb-cb588abefe57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:24.531234 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f66dfeb-7366-45b6-8adb-cb588abefe57 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:24.543020 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1095/2163] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:36:24 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:36:24.547197 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9bca8d-f193-4935-81e6-a46e2b41a438 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:24.557432 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9bca8d-f193-4935-81e6-a46e2b41a438 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:24.562348 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a89bd51-b30e-4b8b-8538-accb80fd3d76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:24.563035 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a89bd51-b30e-4b8b-8538-accb80fd3d76 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:24.566331 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a89bd51-b30e-4b8b-8538-accb80fd3d76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:24.566331 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a89bd51-b30e-4b8b-8538-accb80fd3d76 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:24.566331 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a89bd51-b30e-4b8b-8538-accb80fd3d76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:24.566331 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a89bd51-b30e-4b8b-8538-accb80fd3d76 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:24.576928 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1069/2164] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:36:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:36:24.581343 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f66dfeb-7366-45b6-8adb-cb588abefe57 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:24.592098 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f66dfeb-7366-45b6-8adb-cb588abefe57 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:24.598643 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1240e318-5da1-4568-bd6b-f915b7352c5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:24.600920 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1240e318-5da1-4568-bd6b-f915b7352c5d admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:24.600920 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1240e318-5da1-4568-bd6b-f915b7352c5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:24.600920 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1240e318-5da1-4568-bd6b-f915b7352c5d admin admin] PATH_INFO: `/v3/projects/9ec4c1c1687944b98c1c5cefe05df095/users/d281c92e20d04004a679a8ec65f9e6d5/roles/65feabf3b27944a8b01a86c8ac608457` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:24.621721 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1240e318-5da1-4568-bd6b-f915b7352c5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ec4c1c1687944b98c1c5cefe05df095, user_id=d281c92e20d04004a679a8ec65f9e6d5, role_id=65feabf3b27944a8b01a86c8ac608457)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:24.624540 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1240e318-5da1-4568-bd6b-f915b7352c5d admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:24.690519 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1096/2165] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:36:24 2022] PUT /identity/v3/projects/9ec4c1c1687944b98c1c5cefe05df095/users/d281c92e20d04004a679a8ec65f9e6d5/roles/65feabf3b27944a8b01a86c8ac608457 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:36:24.697267 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb949541-80f1-4c2e-b29e-fa3e12389233 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:24.697267 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb949541-80f1-4c2e-b29e-fa3e12389233 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:24.697820 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb949541-80f1-4c2e-b29e-fa3e12389233 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:24.722053 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-fb949541-80f1-4c2e-b29e-fa3e12389233 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:36:24.723419 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-fb949541-80f1-4c2e-b29e-fa3e12389233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:24.791667 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1070/2166] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:36:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:36:24.801205 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1240e318-5da1-4568-bd6b-f915b7352c5d admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:24.803093 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-1240e318-5da1-4568-bd6b-f915b7352c5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:24.862516 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb949541-80f1-4c2e-b29e-fa3e12389233 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:24.864328 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-fb949541-80f1-4c2e-b29e-fa3e12389233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:24.868044 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1240e318-5da1-4568-bd6b-f915b7352c5d admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:24.871991 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5447b7-eee8-46d1-9f59-8c86f8f0f31d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eaf19a259c2741cea375a38763e33cdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:24.872653 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5447b7-eee8-46d1-9f59-8c86f8f0f31d service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:24.872942 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5447b7-eee8-46d1-9f59-8c86f8f0f31d service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:24.873220 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5447b7-eee8-46d1-9f59-8c86f8f0f31d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:24.883843 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5447b7-eee8-46d1-9f59-8c86f8f0f31d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:24.885498 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5447b7-eee8-46d1-9f59-8c86f8f0f31d service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:24.889458 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb949541-80f1-4c2e-b29e-fa3e12389233 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:24.896130 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bba2c50-84ae-4b7a-bcf7-d9ace2e7f9f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:24.897010 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bba2c50-84ae-4b7a-bcf7-d9ace2e7f9f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:24.897295 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bba2c50-84ae-4b7a-bcf7-d9ace2e7f9f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:24.897571 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bba2c50-84ae-4b7a-bcf7-d9ace2e7f9f9 admin admin] PATH_INFO: `/v3/projects/be6d2e1b8555409ebb608e3f6a5e7ad6` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:24.898905 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bba2c50-84ae-4b7a-bcf7-d9ace2e7f9f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=be6d2e1b8555409ebb608e3f6a5e7ad6)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:24.899182 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1097/2167] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:36:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:36:24.901142 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bba2c50-84ae-4b7a-bcf7-d9ace2e7f9f9 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:25.016367 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-9bba2c50-84ae-4b7a-bcf7-d9ace2e7f9f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be6d2e1b8555409ebb608e3f6a5e7ad6', 'request_id': 'req-9bba2c50-84ae-4b7a-bcf7-d9ace2e7f9f9'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:25.022325 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-9bba2c50-84ae-4b7a-bcf7-d9ace2e7f9f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be6d2e1b8555409ebb608e3f6a5e7ad6', 'request_id': 'req-9bba2c50-84ae-4b7a-bcf7-d9ace2e7f9f9'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:25.029506 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-9bba2c50-84ae-4b7a-bcf7-d9ace2e7f9f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be6d2e1b8555409ebb608e3f6a5e7ad6', 'request_id': 'req-9bba2c50-84ae-4b7a-bcf7-d9ace2e7f9f9'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:25.140333 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-9bba2c50-84ae-4b7a-bcf7-d9ace2e7f9f9 admin admin] The token cache is being invalidate because project be6d2e1b8555409ebb608e3f6a5e7ad6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 02:36:25.144116 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-9bba2c50-84ae-4b7a-bcf7-d9ace2e7f9f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:25.144116 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-9bba2c50-84ae-4b7a-bcf7-d9ace2e7f9f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:25.144116 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1071/2168] 10.222.0.66 () {58 vars in 1232 bytes} [Mon Jul 18 02:36:24 2022] DELETE /identity/v3/projects/be6d2e1b8555409ebb608e3f6a5e7ad6 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 02:36:25.147452 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5447b7-eee8-46d1-9f59-8c86f8f0f31d service neutron] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:25.158408 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5447b7-eee8-46d1-9f59-8c86f8f0f31d service neutron] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:25.218080 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b08eb5-e92d-47e2-9806-1070999bf11e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:25.219180 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b08eb5-e92d-47e2-9806-1070999bf11e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:25.219479 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b08eb5-e92d-47e2-9806-1070999bf11e admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:25.219752 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b08eb5-e92d-47e2-9806-1070999bf11e admin admin] PATH_INFO: `/v3/users/97473baa34c34f25920c7f9d874eb4e8` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:25.221235 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b08eb5-e92d-47e2-9806-1070999bf11e admin admin] RBAC: Authorizing `identity:delete_user(user_id=97473baa34c34f25920c7f9d874eb4e8)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:25.223281 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b08eb5-e92d-47e2-9806-1070999bf11e admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:25.275821 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:36:25.444041 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:36:25.459313 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-16b08eb5-e92d-47e2-9806-1070999bf11e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97473baa34c34f25920c7f9d874eb4e8'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:25.561613 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-16b08eb5-e92d-47e2-9806-1070999bf11e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97473baa34c34f25920c7f9d874eb4e8'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:25.569557 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-16b08eb5-e92d-47e2-9806-1070999bf11e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97473baa34c34f25920c7f9d874eb4e8'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:25.570750 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-16b08eb5-e92d-47e2-9806-1070999bf11e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97473baa34c34f25920c7f9d874eb4e8'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:25.583309 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-16b08eb5-e92d-47e2-9806-1070999bf11e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97473baa34c34f25920c7f9d874eb4e8'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:25.590400 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1098/2169] 10.222.0.66 () {58 vars in 1223 bytes} [Mon Jul 18 02:36:25 2022] DELETE /identity/v3/users/97473baa34c34f25920c7f9d874eb4e8 => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:36:25.735479 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:36:26.047381 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bba2c50-84ae-4b7a-bcf7-d9ace2e7f9f9 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:26.051314 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-9bba2c50-84ae-4b7a-bcf7-d9ace2e7f9f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:26.067819 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bba2c50-84ae-4b7a-bcf7-d9ace2e7f9f9 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:26.129581 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d675c9-732b-4400-89af-06fd74040280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:26.130479 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d675c9-732b-4400-89af-06fd74040280 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:26.130768 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d675c9-732b-4400-89af-06fd74040280 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:26.131045 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d675c9-732b-4400-89af-06fd74040280 admin admin] PATH_INFO: `/v3/projects/17985eb401a34a2ba5c45ce32ba39332` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:26.132235 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d675c9-732b-4400-89af-06fd74040280 admin admin] RBAC: Authorizing `identity:delete_project(project_id=17985eb401a34a2ba5c45ce32ba39332)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:26.134781 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d675c9-732b-4400-89af-06fd74040280 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:26.248720 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-05d675c9-732b-4400-89af-06fd74040280 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17985eb401a34a2ba5c45ce32ba39332', 'request_id': 'req-05d675c9-732b-4400-89af-06fd74040280'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:26.254680 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-05d675c9-732b-4400-89af-06fd74040280 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17985eb401a34a2ba5c45ce32ba39332', 'request_id': 'req-05d675c9-732b-4400-89af-06fd74040280'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:26.261389 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-05d675c9-732b-4400-89af-06fd74040280 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17985eb401a34a2ba5c45ce32ba39332', 'request_id': 'req-05d675c9-732b-4400-89af-06fd74040280'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:26.371695 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-05d675c9-732b-4400-89af-06fd74040280 admin admin] The token cache is being invalidate because project 17985eb401a34a2ba5c45ce32ba39332 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 02:36:26.374130 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-05d675c9-732b-4400-89af-06fd74040280 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:26.374130 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-05d675c9-732b-4400-89af-06fd74040280 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:26.375741 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1072/2170] 10.222.0.66 () {58 vars in 1232 bytes} [Mon Jul 18 02:36:26 2022] DELETE /identity/v3/projects/17985eb401a34a2ba5c45ce32ba39332 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:36:26.381864 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e4bf28-fa1b-4f41-9431-6fc1719cdb16 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:26.382223 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e4bf28-fa1b-4f41-9431-6fc1719cdb16 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:26.382514 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e4bf28-fa1b-4f41-9431-6fc1719cdb16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:26.407399 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-11e4bf28-fa1b-4f41-9431-6fc1719cdb16 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:36:26.408671 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-11e4bf28-fa1b-4f41-9431-6fc1719cdb16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:26.472760 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1099/2171] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:36:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 02:36:26.480845 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e207c3-dd85-4799-8d35-a13f0c188467 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:26.481792 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e207c3-dd85-4799-8d35-a13f0c188467 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:26.482300 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e207c3-dd85-4799-8d35-a13f0c188467 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:26.508299 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-c7e207c3-dd85-4799-8d35-a13f0c188467 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:36:26.509785 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-c7e207c3-dd85-4799-8d35-a13f0c188467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:26.519731 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1073/2172] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:36:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:36:26.523692 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e4bf28-fa1b-4f41-9431-6fc1719cdb16 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:26.534150 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e4bf28-fa1b-4f41-9431-6fc1719cdb16 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:26.540493 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4175834c-29dc-4ffc-ae09-a0ac395ae1f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:26.541354 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4175834c-29dc-4ffc-ae09-a0ac395ae1f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:26.541851 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4175834c-29dc-4ffc-ae09-a0ac395ae1f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:26.542354 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4175834c-29dc-4ffc-ae09-a0ac395ae1f1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:26.543241 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4175834c-29dc-4ffc-ae09-a0ac395ae1f1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:36:26.543845 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4175834c-29dc-4ffc-ae09-a0ac395ae1f1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:26.545642 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4175834c-29dc-4ffc-ae09-a0ac395ae1f1 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:26.559975 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1100/2173] 10.222.0.66 () {58 vars in 1152 bytes} [Mon Jul 18 02:36:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:36:26.564997 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e207c3-dd85-4799-8d35-a13f0c188467 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:26.576048 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e207c3-dd85-4799-8d35-a13f0c188467 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:26.581183 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5da0d45-63b6-46bc-a6d0-e91b06a46e4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:26.581873 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5da0d45-63b6-46bc-a6d0-e91b06a46e4e admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:26.582197 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5da0d45-63b6-46bc-a6d0-e91b06a46e4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:26.582477 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5da0d45-63b6-46bc-a6d0-e91b06a46e4e admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:26.583229 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5da0d45-63b6-46bc-a6d0-e91b06a46e4e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:26.585215 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5da0d45-63b6-46bc-a6d0-e91b06a46e4e admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:26.599232 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:36:26.721153 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1074/2174] 10.222.0.66 () {60 vars in 1152 bytes} [Mon Jul 18 02:36:26 2022] POST /identity/v3/projects => generated 369 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:36:26.721257 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4175834c-29dc-4ffc-ae09-a0ac395ae1f1 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:26.735147 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4175834c-29dc-4ffc-ae09-a0ac395ae1f1 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:26.803173 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccdc73a-a905-48b1-903b-f21c63cd8507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:26.805454 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccdc73a-a905-48b1-903b-f21c63cd8507 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:26.806044 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccdc73a-a905-48b1-903b-f21c63cd8507 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:26.806596 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccdc73a-a905-48b1-903b-f21c63cd8507 admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:26.807522 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccdc73a-a905-48b1-903b-f21c63cd8507 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:26.809782 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccdc73a-a905-48b1-903b-f21c63cd8507 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:26.901607 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:36:26.940774 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1101/2175] 10.222.0.66 () {60 vars in 1143 bytes} [Mon Jul 18 02:36:26 2022] POST /identity/v3/users => generated 285 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 02:36:26.944795 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5da0d45-63b6-46bc-a6d0-e91b06a46e4e admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:26.957416 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5da0d45-63b6-46bc-a6d0-e91b06a46e4e admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:26.964615 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615279a2-1980-4886-886d-8566f446698f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:26.967511 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615279a2-1980-4886-886d-8566f446698f admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:26.967511 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615279a2-1980-4886-886d-8566f446698f admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:26.967511 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615279a2-1980-4886-886d-8566f446698f admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:26.967511 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615279a2-1980-4886-886d-8566f446698f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:26.969350 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615279a2-1980-4886-886d-8566f446698f admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:26.987149 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1075/2176] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:36:26 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:36:26.989658 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccdc73a-a905-48b1-903b-f21c63cd8507 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:27.006046 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccdc73a-a905-48b1-903b-f21c63cd8507 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:27.012235 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4b6f24-f435-4a94-bf89-b7a45aa59e16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:27.012918 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4b6f24-f435-4a94-bf89-b7a45aa59e16 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:27.013207 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4b6f24-f435-4a94-bf89-b7a45aa59e16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:27.013600 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4b6f24-f435-4a94-bf89-b7a45aa59e16 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:27.014369 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4b6f24-f435-4a94-bf89-b7a45aa59e16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:27.016165 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4b6f24-f435-4a94-bf89-b7a45aa59e16 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:27.027806 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1102/2177] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:36:26 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:36:27.031988 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615279a2-1980-4886-886d-8566f446698f admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:27.044734 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615279a2-1980-4886-886d-8566f446698f admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:27.050378 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43cc0c7-bd6c-4359-98f3-b1098047c9fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:27.051463 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43cc0c7-bd6c-4359-98f3-b1098047c9fa admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:27.051836 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43cc0c7-bd6c-4359-98f3-b1098047c9fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:27.052775 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43cc0c7-bd6c-4359-98f3-b1098047c9fa admin admin] PATH_INFO: `/v3/projects/ffa55527bead4570a6f38283af30e012/users/8c4128d673c543d59c83fa603705f45c/roles/65feabf3b27944a8b01a86c8ac608457` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:27.079404 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43cc0c7-bd6c-4359-98f3-b1098047c9fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffa55527bead4570a6f38283af30e012, user_id=8c4128d673c543d59c83fa603705f45c, role_id=65feabf3b27944a8b01a86c8ac608457)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:27.082793 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43cc0c7-bd6c-4359-98f3-b1098047c9fa admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:27.153977 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1076/2178] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:36:27 2022] PUT /identity/v3/projects/ffa55527bead4570a6f38283af30e012/users/8c4128d673c543d59c83fa603705f45c/roles/65feabf3b27944a8b01a86c8ac608457 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:36:27.166921 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621a31e2-60fd-40c6-90a1-b487feb48b7a None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:27.167209 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621a31e2-60fd-40c6-90a1-b487feb48b7a None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:27.167552 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621a31e2-60fd-40c6-90a1-b487feb48b7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:27.197492 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-621a31e2-60fd-40c6-90a1-b487feb48b7a None None] MFA Rules not processed for user `8c4128d673c543d59c83fa603705f45c`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:36:27.198762 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-621a31e2-60fd-40c6-90a1-b487feb48b7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:27.288830 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1103/2179] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:36:27 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 02:36:27.296015 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43cc0c7-bd6c-4359-98f3-b1098047c9fa admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:27.298003 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-f43cc0c7-bd6c-4359-98f3-b1098047c9fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:27.377605 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43cc0c7-bd6c-4359-98f3-b1098047c9fa admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:27.388289 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70161b0-61f4-4ea3-8a07-1a280dbddbe5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1391b81e7234bc18e22b9e84f231256', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:27.389272 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70161b0-61f4-4ea3-8a07-1a280dbddbe5 service glance] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:27.389408 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70161b0-61f4-4ea3-8a07-1a280dbddbe5 service glance] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:27.389676 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70161b0-61f4-4ea3-8a07-1a280dbddbe5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:27.403190 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70161b0-61f4-4ea3-8a07-1a280dbddbe5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:27.405091 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70161b0-61f4-4ea3-8a07-1a280dbddbe5 service glance] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:27.419898 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:36:27.434156 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1077/2180] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:36:27 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:36:27.465034 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:36:27.774565 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:36:28.262268 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:36:29.119049 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:36:29.197293 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:36:29.983368 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:36:30.124861 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:36:30.160301 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:36:30.399198 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:36:30.430023 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88503b39-01c3-4eda-9130-409b8a721c0c None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:30.431320 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88503b39-01c3-4eda-9130-409b8a721c0c None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:30.431420 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88503b39-01c3-4eda-9130-409b8a721c0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:30.468147 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-88503b39-01c3-4eda-9130-409b8a721c0c None None] MFA Rules not processed for user `d281c92e20d04004a679a8ec65f9e6d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:36:30.469392 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-88503b39-01c3-4eda-9130-409b8a721c0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:30.556909 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1104/2181] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:36:30 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:36:30.565603 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64553c64-cb65-42c1-a6c3-0834dd3c7a94 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:30.565899 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64553c64-cb65-42c1-a6c3-0834dd3c7a94 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:30.566218 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64553c64-cb65-42c1-a6c3-0834dd3c7a94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:30.591302 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-64553c64-cb65-42c1-a6c3-0834dd3c7a94 None None] MFA Rules not processed for user `d281c92e20d04004a679a8ec65f9e6d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:36:30.591302 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-64553c64-cb65-42c1-a6c3-0834dd3c7a94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:30.596724 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1078/2182] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:36:30 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:36:30.611176 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88503b39-01c3-4eda-9130-409b8a721c0c None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:30.613013 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-88503b39-01c3-4eda-9130-409b8a721c0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:30.618707 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:36:30.707148 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88503b39-01c3-4eda-9130-409b8a721c0c None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:30.713536 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbd43bd-4fb0-4d5b-95a1-6ef676862273 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80b138831bc54c7b986d50092d62d4af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:30.715916 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbd43bd-4fb0-4d5b-95a1-6ef676862273 service nova] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:30.715916 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbd43bd-4fb0-4d5b-95a1-6ef676862273 service nova] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:30.715916 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbd43bd-4fb0-4d5b-95a1-6ef676862273 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:30.723343 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbd43bd-4fb0-4d5b-95a1-6ef676862273 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:30.724948 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbd43bd-4fb0-4d5b-95a1-6ef676862273 service nova] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:30.736681 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1105/2183] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:36:30 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:36:31.147806 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:36:31.543394 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9a8440-35c9-4704-91b8-8d4d666d226e None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:31.543686 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9a8440-35c9-4704-91b8-8d4d666d226e None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:31.543956 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9a8440-35c9-4704-91b8-8d4d666d226e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:31.563692 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-cc9a8440-35c9-4704-91b8-8d4d666d226e None None] MFA Rules not processed for user `d281c92e20d04004a679a8ec65f9e6d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:36:31.565061 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-cc9a8440-35c9-4704-91b8-8d4d666d226e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:31.575153 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1079/2184] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:36:31 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:36:31.669823 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:36:33.146222 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:36:33.662806 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:36:34.628510 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:36:34.752005 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:36:34.840376 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=93,keystone:DELETE=2,keystone:INSERT=5 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:36:34.869547 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=93,keystone:DELETE=2,keystone:INSERT=5 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:36:35.012830 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:36:35.095035 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:36:35.472640 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:36:35.589434 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:36:36.352565 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:36:36.410443 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:36:37.388836 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:36:37.413383 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:36:37.821441 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:36:37.891956 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=90,keystone:INSERT=4,keystone:DELETE=6 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:36:38.006286 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:36:38.029173 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:36:38.154963 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:36:38.178275 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:36:38.251974 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:36:38.272744 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:36:38.387193 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:36:38.409891 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:36:38.509259 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:36:38.538074 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:36:38.560808 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:36:38.640870 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:36:38.658010 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:36:38.920728 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:36:38.995881 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:36:39.075164 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:36:39.177808 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:36:48.903648 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=62 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:36:49.114169 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbd43bd-4fb0-4d5b-95a1-6ef676862273 service nova] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:49.115598 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-5fbd43bd-4fb0-4d5b-95a1-6ef676862273 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:49.182524 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbd43bd-4fb0-4d5b-95a1-6ef676862273 service nova] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:49.187827 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04191c0-b671-4186-8fc6-e6acac0062f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:49.188744 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04191c0-b671-4186-8fc6-e6acac0062f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:49.189030 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04191c0-b671-4186-8fc6-e6acac0062f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:49.189309 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04191c0-b671-4186-8fc6-e6acac0062f5 admin admin] PATH_INFO: `/v3/users/2b858f9e4b474eaa918623a9148f28b8` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:49.190585 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04191c0-b671-4186-8fc6-e6acac0062f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b858f9e4b474eaa918623a9148f28b8)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:49.192549 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04191c0-b671-4186-8fc6-e6acac0062f5 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:49.331745 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=4 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:36:49.399801 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-e04191c0-b671-4186-8fc6-e6acac0062f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b858f9e4b474eaa918623a9148f28b8'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:49.471687 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-e04191c0-b671-4186-8fc6-e6acac0062f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b858f9e4b474eaa918623a9148f28b8'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:49.479787 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-e04191c0-b671-4186-8fc6-e6acac0062f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b858f9e4b474eaa918623a9148f28b8'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:49.480563 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-e04191c0-b671-4186-8fc6-e6acac0062f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b858f9e4b474eaa918623a9148f28b8'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:49.493245 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-e04191c0-b671-4186-8fc6-e6acac0062f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b858f9e4b474eaa918623a9148f28b8'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:49.499487 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1106/2185] 10.222.0.66 () {58 vars in 1223 bytes} [Mon Jul 18 02:36:49 2022] DELETE /identity/v3/users/2b858f9e4b474eaa918623a9148f28b8 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:36:49.504154 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6795e5-c0eb-40c9-b499-d34c318bb782 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:49.504527 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6795e5-c0eb-40c9-b499-d34c318bb782 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:49.504813 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6795e5-c0eb-40c9-b499-d34c318bb782 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:49.523180 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-2f6795e5-c0eb-40c9-b499-d34c318bb782 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:36:49.524659 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-2f6795e5-c0eb-40c9-b499-d34c318bb782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:49.585693 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1080/2186] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:36:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:36:49.596280 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04191c0-b671-4186-8fc6-e6acac0062f5 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:49.597700 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-e04191c0-b671-4186-8fc6-e6acac0062f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:49.658668 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04191c0-b671-4186-8fc6-e6acac0062f5 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:49.662415 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c73ca09-0574-46c3-96d8-dcb933d08f60 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eaf19a259c2741cea375a38763e33cdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:49.663099 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c73ca09-0574-46c3-96d8-dcb933d08f60 service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:49.663386 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c73ca09-0574-46c3-96d8-dcb933d08f60 service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:49.663665 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c73ca09-0574-46c3-96d8-dcb933d08f60 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:49.674814 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c73ca09-0574-46c3-96d8-dcb933d08f60 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:49.676303 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c73ca09-0574-46c3-96d8-dcb933d08f60 service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:49.687976 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1107/2187] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:36:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:36:50.201039 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6795e5-c0eb-40c9-b499-d34c318bb782 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:50.202592 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-2f6795e5-c0eb-40c9-b499-d34c318bb782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:50.215172 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6795e5-c0eb-40c9-b499-d34c318bb782 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:50.220711 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f58ad4-df5c-489c-a618-e1700e38e3bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:50.221646 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f58ad4-df5c-489c-a618-e1700e38e3bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:50.221935 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f58ad4-df5c-489c-a618-e1700e38e3bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:50.222316 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f58ad4-df5c-489c-a618-e1700e38e3bb admin admin] PATH_INFO: `/v3/projects/5d84de9074bb4d9a87d20326924df7e6` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:50.223681 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f58ad4-df5c-489c-a618-e1700e38e3bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d84de9074bb4d9a87d20326924df7e6)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:50.225538 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f58ad4-df5c-489c-a618-e1700e38e3bb admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:50.362213 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-a6f58ad4-df5c-489c-a618-e1700e38e3bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d84de9074bb4d9a87d20326924df7e6', 'request_id': 'req-a6f58ad4-df5c-489c-a618-e1700e38e3bb'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:50.368581 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-a6f58ad4-df5c-489c-a618-e1700e38e3bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d84de9074bb4d9a87d20326924df7e6', 'request_id': 'req-a6f58ad4-df5c-489c-a618-e1700e38e3bb'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:50.375642 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-a6f58ad4-df5c-489c-a618-e1700e38e3bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d84de9074bb4d9a87d20326924df7e6', 'request_id': 'req-a6f58ad4-df5c-489c-a618-e1700e38e3bb'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:50.489241 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-a6f58ad4-df5c-489c-a618-e1700e38e3bb admin admin] The token cache is being invalidate because project 5d84de9074bb4d9a87d20326924df7e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 02:36:50.489480 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-a6f58ad4-df5c-489c-a618-e1700e38e3bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:50.490137 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-a6f58ad4-df5c-489c-a618-e1700e38e3bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:50.492032 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1081/2188] 10.222.0.66 () {58 vars in 1232 bytes} [Mon Jul 18 02:36:50 2022] DELETE /identity/v3/projects/5d84de9074bb4d9a87d20326924df7e6 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:36:50.497112 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6e1bc2-af35-4bcb-b67a-51b84438b233 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:50.497537 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6e1bc2-af35-4bcb-b67a-51b84438b233 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:50.497640 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6e1bc2-af35-4bcb-b67a-51b84438b233 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:50.519474 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-dd6e1bc2-af35-4bcb-b67a-51b84438b233 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:36:50.526218 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-dd6e1bc2-af35-4bcb-b67a-51b84438b233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:50.539462 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:36:50.587811 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1108/2189] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:36:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:36:50.597386 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05952154-daa3-4c39-9b69-91ea816e6111 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:50.597870 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05952154-daa3-4c39-9b69-91ea816e6111 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:50.598217 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05952154-daa3-4c39-9b69-91ea816e6111 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:50.618283 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-05952154-daa3-4c39-9b69-91ea816e6111 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:36:50.619594 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-05952154-daa3-4c39-9b69-91ea816e6111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:50.633965 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1082/2190] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:36:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:36:50.637135 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6e1bc2-af35-4bcb-b67a-51b84438b233 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:50.647913 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6e1bc2-af35-4bcb-b67a-51b84438b233 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:50.653047 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a683a3-9c32-4802-b8a4-6f42ac70f324 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:50.654111 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a683a3-9c32-4802-b8a4-6f42ac70f324 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:50.654111 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a683a3-9c32-4802-b8a4-6f42ac70f324 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:50.654327 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a683a3-9c32-4802-b8a4-6f42ac70f324 admin admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:50.655058 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a683a3-9c32-4802-b8a4-6f42ac70f324 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:36:50.655369 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a683a3-9c32-4802-b8a4-6f42ac70f324 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:50.657012 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a683a3-9c32-4802-b8a4-6f42ac70f324 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:50.673188 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1109/2191] 10.222.0.66 () {58 vars in 1152 bytes} [Mon Jul 18 02:36:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:36:50.677415 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05952154-daa3-4c39-9b69-91ea816e6111 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:50.688477 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05952154-daa3-4c39-9b69-91ea816e6111 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:50.693251 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb8e769-09f1-4472-90c8-38e1b5600e74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:50.693916 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb8e769-09f1-4472-90c8-38e1b5600e74 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:50.694242 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb8e769-09f1-4472-90c8-38e1b5600e74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:50.694520 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb8e769-09f1-4472-90c8-38e1b5600e74 admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:50.695284 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb8e769-09f1-4472-90c8-38e1b5600e74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:50.697290 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb8e769-09f1-4472-90c8-38e1b5600e74 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:50.845141 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1083/2192] 10.222.0.66 () {60 vars in 1152 bytes} [Mon Jul 18 02:36:50 2022] POST /identity/v3/projects => generated 371 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:36:50.849466 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a683a3-9c32-4802-b8a4-6f42ac70f324 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:50.860881 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a683a3-9c32-4802-b8a4-6f42ac70f324 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:50.920659 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf138150-9f19-4b63-9cc7-c32bf9b0b2d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:50.921361 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf138150-9f19-4b63-9cc7-c32bf9b0b2d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:50.921644 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf138150-9f19-4b63-9cc7-c32bf9b0b2d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:50.921912 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf138150-9f19-4b63-9cc7-c32bf9b0b2d9 admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:50.922642 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf138150-9f19-4b63-9cc7-c32bf9b0b2d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:50.924581 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf138150-9f19-4b63-9cc7-c32bf9b0b2d9 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:51.053556 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1110/2193] 10.222.0.66 () {60 vars in 1143 bytes} [Mon Jul 18 02:36:50 2022] POST /identity/v3/users => generated 286 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:36:51.057640 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb8e769-09f1-4472-90c8-38e1b5600e74 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:51.068996 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb8e769-09f1-4472-90c8-38e1b5600e74 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:51.073558 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caddd748-e20b-4cb9-ba5a-c6c4d850d751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:51.074196 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caddd748-e20b-4cb9-ba5a-c6c4d850d751 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:51.074483 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caddd748-e20b-4cb9-ba5a-c6c4d850d751 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:51.074918 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caddd748-e20b-4cb9-ba5a-c6c4d850d751 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:51.075645 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caddd748-e20b-4cb9-ba5a-c6c4d850d751 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:51.077252 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caddd748-e20b-4cb9-ba5a-c6c4d850d751 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:51.088423 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1084/2194] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:36:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:36:51.092478 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf138150-9f19-4b63-9cc7-c32bf9b0b2d9 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:51.102515 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf138150-9f19-4b63-9cc7-c32bf9b0b2d9 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:51.108184 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedcb6e3-e7fb-4ff9-9561-f4bcfd7b5435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:51.108848 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedcb6e3-e7fb-4ff9-9561-f4bcfd7b5435 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:51.109137 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedcb6e3-e7fb-4ff9-9561-f4bcfd7b5435 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:51.109448 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedcb6e3-e7fb-4ff9-9561-f4bcfd7b5435 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:51.110246 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedcb6e3-e7fb-4ff9-9561-f4bcfd7b5435 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:51.111902 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedcb6e3-e7fb-4ff9-9561-f4bcfd7b5435 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:51.122794 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1111/2195] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:36:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:36:51.127085 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caddd748-e20b-4cb9-ba5a-c6c4d850d751 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:51.136953 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caddd748-e20b-4cb9-ba5a-c6c4d850d751 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:51.141485 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01414a2-9cd4-4c9f-9cb9-42f6baeed087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:51.142405 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01414a2-9cd4-4c9f-9cb9-42f6baeed087 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:51.142685 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01414a2-9cd4-4c9f-9cb9-42f6baeed087 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:51.142952 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01414a2-9cd4-4c9f-9cb9-42f6baeed087 admin admin] PATH_INFO: `/v3/projects/d615520de8a64205981c88430498f5f6/users/86d9949aaf5749c9b00e1e04171e6f93/roles/65feabf3b27944a8b01a86c8ac608457` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:51.157711 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01414a2-9cd4-4c9f-9cb9-42f6baeed087 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d615520de8a64205981c88430498f5f6, user_id=86d9949aaf5749c9b00e1e04171e6f93, role_id=65feabf3b27944a8b01a86c8ac608457)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:51.160901 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01414a2-9cd4-4c9f-9cb9-42f6baeed087 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:51.269323 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1085/2196] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:36:51 2022] PUT /identity/v3/projects/d615520de8a64205981c88430498f5f6/users/86d9949aaf5749c9b00e1e04171e6f93/roles/65feabf3b27944a8b01a86c8ac608457 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:36:51.274840 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7236ce-9e9f-4ae2-8fa1-55b54e7dd3f3 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:51.275134 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7236ce-9e9f-4ae2-8fa1-55b54e7dd3f3 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:51.275404 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7236ce-9e9f-4ae2-8fa1-55b54e7dd3f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:51.297256 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-7a7236ce-9e9f-4ae2-8fa1-55b54e7dd3f3 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:36:51.298633 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-7a7236ce-9e9f-4ae2-8fa1-55b54e7dd3f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:51.356797 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1112/2197] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:36:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:36:51.366608 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01414a2-9cd4-4c9f-9cb9-42f6baeed087 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:51.368426 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-f01414a2-9cd4-4c9f-9cb9-42f6baeed087 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:51.392869 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:36:51.426737 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01414a2-9cd4-4c9f-9cb9-42f6baeed087 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:51.430579 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4217013-edf6-44ad-a570-befb341c73aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eaf19a259c2741cea375a38763e33cdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:51.431224 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4217013-edf6-44ad-a570-befb341c73aa service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:51.431509 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4217013-edf6-44ad-a570-befb341c73aa service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:51.431792 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4217013-edf6-44ad-a570-befb341c73aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:51.441811 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4217013-edf6-44ad-a570-befb341c73aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:51.443274 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4217013-edf6-44ad-a570-befb341c73aa service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:51.453089 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1086/2198] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:36:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:36:53.027476 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f569112b-dd47-43fe-ad34-2399c7f070a3 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:53.028491 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f569112b-dd47-43fe-ad34-2399c7f070a3 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:53.028607 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f569112b-dd47-43fe-ad34-2399c7f070a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:53.055940 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-f569112b-dd47-43fe-ad34-2399c7f070a3 None None] MFA Rules not processed for user `86d9949aaf5749c9b00e1e04171e6f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:36:53.057280 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-f569112b-dd47-43fe-ad34-2399c7f070a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:53.075742 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:36:53.124797 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1113/2199] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:36:53 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 02:36:53.135676 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4217013-edf6-44ad-a570-befb341c73aa service neutron] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:53.137284 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-c4217013-edf6-44ad-a570-befb341c73aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:53.196446 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4217013-edf6-44ad-a570-befb341c73aa service neutron] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:53.201330 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56e9a76-8d0d-46e0-92a5-c3572663a5fa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd49388facdf446885e55b45ef91f01e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:53.201817 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f569112b-dd47-43fe-ad34-2399c7f070a3 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:53.201997 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56e9a76-8d0d-46e0-92a5-c3572663a5fa service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:53.202314 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56e9a76-8d0d-46e0-92a5-c3572663a5fa service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:53.202598 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56e9a76-8d0d-46e0-92a5-c3572663a5fa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:53.203462 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-f569112b-dd47-43fe-ad34-2399c7f070a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:53.211876 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56e9a76-8d0d-46e0-92a5-c3572663a5fa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:53.215049 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56e9a76-8d0d-46e0-92a5-c3572663a5fa service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:53.220100 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f569112b-dd47-43fe-ad34-2399c7f070a3 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:53.223560 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1087/2200] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:36:53 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 02:36:53.225105 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bc049a-a152-421b-83e1-d191e0b1e9f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:53.226013 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bc049a-a152-421b-83e1-d191e0b1e9f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:53.226423 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bc049a-a152-421b-83e1-d191e0b1e9f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:53.226806 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bc049a-a152-421b-83e1-d191e0b1e9f0 admin admin] PATH_INFO: `/v3/users/5cd0ca9fce844c748c9c562be96f9553` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:53.228093 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bc049a-a152-421b-83e1-d191e0b1e9f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5cd0ca9fce844c748c9c562be96f9553)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:53.230323 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bc049a-a152-421b-83e1-d191e0b1e9f0 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:53.475148 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-d4bc049a-a152-421b-83e1-d191e0b1e9f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5cd0ca9fce844c748c9c562be96f9553'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:53.546422 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-d4bc049a-a152-421b-83e1-d191e0b1e9f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5cd0ca9fce844c748c9c562be96f9553'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:53.554011 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-d4bc049a-a152-421b-83e1-d191e0b1e9f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5cd0ca9fce844c748c9c562be96f9553'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:53.554711 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-d4bc049a-a152-421b-83e1-d191e0b1e9f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5cd0ca9fce844c748c9c562be96f9553'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:53.567141 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-d4bc049a-a152-421b-83e1-d191e0b1e9f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5cd0ca9fce844c748c9c562be96f9553'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:53.573235 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1114/2201] 10.222.0.66 () {58 vars in 1223 bytes} [Mon Jul 18 02:36:53 2022] DELETE /identity/v3/users/5cd0ca9fce844c748c9c562be96f9553 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 02:36:53.577941 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68239bdd-cf04-4e4d-8325-6857a877a3a9 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:53.578354 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68239bdd-cf04-4e4d-8325-6857a877a3a9 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:53.578640 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68239bdd-cf04-4e4d-8325-6857a877a3a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:53.596508 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-68239bdd-cf04-4e4d-8325-6857a877a3a9 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:36:53.597752 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-68239bdd-cf04-4e4d-8325-6857a877a3a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:53.660738 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1088/2202] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:36:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:36:53.671347 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bc049a-a152-421b-83e1-d191e0b1e9f0 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:53.672789 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-d4bc049a-a152-421b-83e1-d191e0b1e9f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:53.730538 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bc049a-a152-421b-83e1-d191e0b1e9f0 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:53.734905 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6338b31-15b1-4fff-8f2a-c978bcba2f1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eaf19a259c2741cea375a38763e33cdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:53.735551 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6338b31-15b1-4fff-8f2a-c978bcba2f1b service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:53.735838 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6338b31-15b1-4fff-8f2a-c978bcba2f1b service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:53.736115 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6338b31-15b1-4fff-8f2a-c978bcba2f1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:53.745608 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6338b31-15b1-4fff-8f2a-c978bcba2f1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:53.747204 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6338b31-15b1-4fff-8f2a-c978bcba2f1b service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:53.757020 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1115/2203] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:36:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:36:54.369034 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68239bdd-cf04-4e4d-8325-6857a877a3a9 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:54.374620 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-68239bdd-cf04-4e4d-8325-6857a877a3a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:54.383005 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68239bdd-cf04-4e4d-8325-6857a877a3a9 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:54.388319 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17237818-171e-40ad-b338-d0b1b66fc759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:54.389347 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17237818-171e-40ad-b338-d0b1b66fc759 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:54.389630 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17237818-171e-40ad-b338-d0b1b66fc759 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:54.389904 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17237818-171e-40ad-b338-d0b1b66fc759 admin admin] PATH_INFO: `/v3/projects/a784b54e7089433cb4918d39f18a9ced` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:54.391143 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17237818-171e-40ad-b338-d0b1b66fc759 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a784b54e7089433cb4918d39f18a9ced)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:54.393061 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17237818-171e-40ad-b338-d0b1b66fc759 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:54.507394 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-17237818-171e-40ad-b338-d0b1b66fc759 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a784b54e7089433cb4918d39f18a9ced', 'request_id': 'req-17237818-171e-40ad-b338-d0b1b66fc759'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:54.513703 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-17237818-171e-40ad-b338-d0b1b66fc759 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a784b54e7089433cb4918d39f18a9ced', 'request_id': 'req-17237818-171e-40ad-b338-d0b1b66fc759'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:54.515561 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:36:54.520021 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-17237818-171e-40ad-b338-d0b1b66fc759 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a784b54e7089433cb4918d39f18a9ced', 'request_id': 'req-17237818-171e-40ad-b338-d0b1b66fc759'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:54.627908 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-17237818-171e-40ad-b338-d0b1b66fc759 admin admin] The token cache is being invalidate because project a784b54e7089433cb4918d39f18a9ced was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 02:36:54.633169 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-17237818-171e-40ad-b338-d0b1b66fc759 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:54.633169 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-17237818-171e-40ad-b338-d0b1b66fc759 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:36:54.633169 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1089/2204] 10.222.0.66 () {58 vars in 1232 bytes} [Mon Jul 18 02:36:54 2022] DELETE /identity/v3/projects/a784b54e7089433cb4918d39f18a9ced => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:36:54.636971 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7956e4ed-4bd5-43d5-86dc-42940990a8c9 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:54.637486 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7956e4ed-4bd5-43d5-86dc-42940990a8c9 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:54.637578 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7956e4ed-4bd5-43d5-86dc-42940990a8c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:54.656347 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-7956e4ed-4bd5-43d5-86dc-42940990a8c9 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:36:54.657589 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-7956e4ed-4bd5-43d5-86dc-42940990a8c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:54.690873 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:36:54.724690 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1116/2205] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:36:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:36:54.733877 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d614fec-a9a0-4ef9-8160-4ea82c5ab823 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:54.734196 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d614fec-a9a0-4ef9-8160-4ea82c5ab823 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:54.734476 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d614fec-a9a0-4ef9-8160-4ea82c5ab823 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:54.754471 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-5d614fec-a9a0-4ef9-8160-4ea82c5ab823 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:36:54.755819 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-5d614fec-a9a0-4ef9-8160-4ea82c5ab823 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:54.769052 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1090/2206] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:36:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 02:36:54.770819 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7956e4ed-4bd5-43d5-86dc-42940990a8c9 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:54.780412 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7956e4ed-4bd5-43d5-86dc-42940990a8c9 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:54.786014 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3336b12d-9ae0-4e42-89d5-a8a651fb32a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:54.786291 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3336b12d-9ae0-4e42-89d5-a8a651fb32a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:54.787149 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3336b12d-9ae0-4e42-89d5-a8a651fb32a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:54.787442 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3336b12d-9ae0-4e42-89d5-a8a651fb32a5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:54.788178 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3336b12d-9ae0-4e42-89d5-a8a651fb32a5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:36:54.788487 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3336b12d-9ae0-4e42-89d5-a8a651fb32a5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:54.790113 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3336b12d-9ae0-4e42-89d5-a8a651fb32a5 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:54.803978 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1117/2207] 10.222.0.66 () {58 vars in 1152 bytes} [Mon Jul 18 02:36:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 02:36:54.811085 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d614fec-a9a0-4ef9-8160-4ea82c5ab823 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:54.820578 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d614fec-a9a0-4ef9-8160-4ea82c5ab823 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:54.825516 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27526704-b2bd-4c1e-9ad3-84f0710500ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:54.826222 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27526704-b2bd-4c1e-9ad3-84f0710500ce admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:54.826512 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27526704-b2bd-4c1e-9ad3-84f0710500ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:54.826793 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27526704-b2bd-4c1e-9ad3-84f0710500ce admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:54.827489 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27526704-b2bd-4c1e-9ad3-84f0710500ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:54.829427 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27526704-b2bd-4c1e-9ad3-84f0710500ce admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:54.925258 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1091/2208] 10.222.0.66 () {60 vars in 1152 bytes} [Mon Jul 18 02:36:54 2022] POST /identity/v3/projects => generated 363 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:36:54.931029 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3336b12d-9ae0-4e42-89d5-a8a651fb32a5 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:54.955795 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3336b12d-9ae0-4e42-89d5-a8a651fb32a5 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:55.033156 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cec799-e8a2-47f4-ae17-1cd9a4f5960a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:55.033806 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cec799-e8a2-47f4-ae17-1cd9a4f5960a admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:55.034123 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cec799-e8a2-47f4-ae17-1cd9a4f5960a admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:55.034400 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cec799-e8a2-47f4-ae17-1cd9a4f5960a admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:55.035200 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cec799-e8a2-47f4-ae17-1cd9a4f5960a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:55.037450 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cec799-e8a2-47f4-ae17-1cd9a4f5960a admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:55.127951 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1118/2209] 10.222.0.66 () {60 vars in 1143 bytes} [Mon Jul 18 02:36:54 2022] POST /identity/v3/users => generated 282 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:36:55.129238 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27526704-b2bd-4c1e-9ad3-84f0710500ce admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:55.141285 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27526704-b2bd-4c1e-9ad3-84f0710500ce admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:55.146305 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f15732b-78c5-42f7-a7cf-c264a932bb16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:55.147183 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f15732b-78c5-42f7-a7cf-c264a932bb16 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:55.147467 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f15732b-78c5-42f7-a7cf-c264a932bb16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:55.147742 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f15732b-78c5-42f7-a7cf-c264a932bb16 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:55.148497 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f15732b-78c5-42f7-a7cf-c264a932bb16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:55.150511 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f15732b-78c5-42f7-a7cf-c264a932bb16 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:55.162798 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1092/2210] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:36:55 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 02:36:55.167119 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cec799-e8a2-47f4-ae17-1cd9a4f5960a admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:55.177942 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cec799-e8a2-47f4-ae17-1cd9a4f5960a admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:55.187916 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc4c6e4-b308-4e94-b1ea-6c5f86f3dee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:55.188607 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc4c6e4-b308-4e94-b1ea-6c5f86f3dee8 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:55.188915 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc4c6e4-b308-4e94-b1ea-6c5f86f3dee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:55.189192 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc4c6e4-b308-4e94-b1ea-6c5f86f3dee8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:55.189925 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc4c6e4-b308-4e94-b1ea-6c5f86f3dee8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:55.191858 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc4c6e4-b308-4e94-b1ea-6c5f86f3dee8 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:55.204308 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1119/2211] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:36:55 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:36:55.209094 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f15732b-78c5-42f7-a7cf-c264a932bb16 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:55.221912 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f15732b-78c5-42f7-a7cf-c264a932bb16 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:55.226668 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd06b581-18ce-4f60-8832-84f53b981e31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:55.227556 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd06b581-18ce-4f60-8832-84f53b981e31 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:55.227837 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd06b581-18ce-4f60-8832-84f53b981e31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:55.228110 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd06b581-18ce-4f60-8832-84f53b981e31 admin admin] PATH_INFO: `/v3/projects/2f35aa2ec35f41448e63643c5c043ea8/users/994a19636bb348ce9fdb015391dca6f1/roles/65feabf3b27944a8b01a86c8ac608457` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:55.246656 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd06b581-18ce-4f60-8832-84f53b981e31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f35aa2ec35f41448e63643c5c043ea8, user_id=994a19636bb348ce9fdb015391dca6f1, role_id=65feabf3b27944a8b01a86c8ac608457)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:55.250228 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd06b581-18ce-4f60-8832-84f53b981e31 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:55.374104 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1093/2212] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:36:55 2022] PUT /identity/v3/projects/2f35aa2ec35f41448e63643c5c043ea8/users/994a19636bb348ce9fdb015391dca6f1/roles/65feabf3b27944a8b01a86c8ac608457 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:36:55.385114 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136aef72-ec68-470c-8c43-3f21076964cf None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:55.385336 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136aef72-ec68-470c-8c43-3f21076964cf None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:55.385615 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136aef72-ec68-470c-8c43-3f21076964cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:55.414259 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-136aef72-ec68-470c-8c43-3f21076964cf None None] MFA Rules not processed for user `994a19636bb348ce9fdb015391dca6f1`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:36:55.415476 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-136aef72-ec68-470c-8c43-3f21076964cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:55.496096 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1120/2213] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:36:55 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:36:55.503998 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd06b581-18ce-4f60-8832-84f53b981e31 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:36:55.505598 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-fd06b581-18ce-4f60-8832-84f53b981e31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:55.568495 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd06b581-18ce-4f60-8832-84f53b981e31 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:36:55.572201 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60de63be-7ee3-4cc9-9f84-8402ec87b3bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd49388facdf446885e55b45ef91f01e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:36:55.572849 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60de63be-7ee3-4cc9-9f84-8402ec87b3bc service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:55.573136 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60de63be-7ee3-4cc9-9f84-8402ec87b3bc service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:55.573426 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60de63be-7ee3-4cc9-9f84-8402ec87b3bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:55.582618 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60de63be-7ee3-4cc9-9f84-8402ec87b3bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:36:55.584060 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60de63be-7ee3-4cc9-9f84-8402ec87b3bc service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:36:55.593721 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1094/2214] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:36:55 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:36:56.109332 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83f4cd3-456a-424d-8024-eeaa3dfd4fa5 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:36:56.109843 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83f4cd3-456a-424d-8024-eeaa3dfd4fa5 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:36:56.109939 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83f4cd3-456a-424d-8024-eeaa3dfd4fa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:36:56.127813 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-f83f4cd3-456a-424d-8024-eeaa3dfd4fa5 None None] MFA Rules not processed for user `86d9949aaf5749c9b00e1e04171e6f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:36:56.129007 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-f83f4cd3-456a-424d-8024-eeaa3dfd4fa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:36:56.155573 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:36:56.186798 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1121/2215] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:36:56 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:37:05.588853 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=56,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:37:06.344246 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=12 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:37:22.139754 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30de9704-27ad-4090-981f-bdedaa40c8e9 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:37:22.140279 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30de9704-27ad-4090-981f-bdedaa40c8e9 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:37:22.140387 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30de9704-27ad-4090-981f-bdedaa40c8e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:37:22.159016 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-30de9704-27ad-4090-981f-bdedaa40c8e9 None None] MFA Rules not processed for user `94c16391d39f4498a55c89bf2838788f`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:37:22.160252 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-30de9704-27ad-4090-981f-bdedaa40c8e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:37:22.214990 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1095/2216] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:37:22 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:37:23.785941 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83f4cd3-456a-424d-8024-eeaa3dfd4fa5 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:37:23.787615 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-f83f4cd3-456a-424d-8024-eeaa3dfd4fa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:37:23.883263 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83f4cd3-456a-424d-8024-eeaa3dfd4fa5 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:37:23.892635 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e26cbde-c60a-45e2-9d22-88f432767f6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:37:23.893943 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e26cbde-c60a-45e2-9d22-88f432767f6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:37:23.894273 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e26cbde-c60a-45e2-9d22-88f432767f6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:37:23.894558 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e26cbde-c60a-45e2-9d22-88f432767f6b admin admin] PATH_INFO: `/v3/users/8c4128d673c543d59c83fa603705f45c` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:37:23.899910 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e26cbde-c60a-45e2-9d22-88f432767f6b admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c4128d673c543d59c83fa603705f45c)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:37:23.899910 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e26cbde-c60a-45e2-9d22-88f432767f6b admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:37:24.112193 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-6e26cbde-c60a-45e2-9d22-88f432767f6b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c4128d673c543d59c83fa603705f45c'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:37:24.190231 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-6e26cbde-c60a-45e2-9d22-88f432767f6b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c4128d673c543d59c83fa603705f45c'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:37:24.199523 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-6e26cbde-c60a-45e2-9d22-88f432767f6b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c4128d673c543d59c83fa603705f45c'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:37:24.200297 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-6e26cbde-c60a-45e2-9d22-88f432767f6b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c4128d673c543d59c83fa603705f45c'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:37:24.214258 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-6e26cbde-c60a-45e2-9d22-88f432767f6b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c4128d673c543d59c83fa603705f45c'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:37:24.223786 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1122/2217] 10.222.0.66 () {58 vars in 1223 bytes} [Mon Jul 18 02:37:23 2022] DELETE /identity/v3/users/8c4128d673c543d59c83fa603705f45c => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:37:24.228637 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc08211a-9df0-4914-ad5e-e55c4aada4ea None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:37:24.229090 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc08211a-9df0-4914-ad5e-e55c4aada4ea None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:37:24.229205 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc08211a-9df0-4914-ad5e-e55c4aada4ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:37:24.248641 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-bc08211a-9df0-4914-ad5e-e55c4aada4ea None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:37:24.249850 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-bc08211a-9df0-4914-ad5e-e55c4aada4ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:37:24.310115 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1096/2218] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:37:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:37:24.319572 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e26cbde-c60a-45e2-9d22-88f432767f6b admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:37:24.321115 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-6e26cbde-c60a-45e2-9d22-88f432767f6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:37:24.399374 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e26cbde-c60a-45e2-9d22-88f432767f6b admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:37:24.403973 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a36de39-4eb7-429f-aaa7-7a3abef1bf86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eaf19a259c2741cea375a38763e33cdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:37:24.404369 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a36de39-4eb7-429f-aaa7-7a3abef1bf86 service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:37:24.404653 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a36de39-4eb7-429f-aaa7-7a3abef1bf86 service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:37:24.404924 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a36de39-4eb7-429f-aaa7-7a3abef1bf86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:37:24.419940 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a36de39-4eb7-429f-aaa7-7a3abef1bf86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:37:24.422600 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a36de39-4eb7-429f-aaa7-7a3abef1bf86 service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:37:24.435509 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1123/2219] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:37:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:37:24.891447 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc08211a-9df0-4914-ad5e-e55c4aada4ea None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:37:24.894979 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-bc08211a-9df0-4914-ad5e-e55c4aada4ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:37:24.913107 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc08211a-9df0-4914-ad5e-e55c4aada4ea None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:37:24.923412 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb39fa79-abef-48e6-8d56-96091db7890b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:37:24.924805 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb39fa79-abef-48e6-8d56-96091db7890b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:37:24.925811 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb39fa79-abef-48e6-8d56-96091db7890b admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:37:24.926311 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb39fa79-abef-48e6-8d56-96091db7890b admin admin] PATH_INFO: `/v3/projects/ffa55527bead4570a6f38283af30e012` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:37:24.929960 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb39fa79-abef-48e6-8d56-96091db7890b admin admin] RBAC: Authorizing `identity:delete_project(project_id=ffa55527bead4570a6f38283af30e012)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:37:24.932170 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb39fa79-abef-48e6-8d56-96091db7890b admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:37:25.091840 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-cb39fa79-abef-48e6-8d56-96091db7890b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ffa55527bead4570a6f38283af30e012', 'request_id': 'req-cb39fa79-abef-48e6-8d56-96091db7890b'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:37:25.097749 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-cb39fa79-abef-48e6-8d56-96091db7890b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ffa55527bead4570a6f38283af30e012', 'request_id': 'req-cb39fa79-abef-48e6-8d56-96091db7890b'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:37:25.104571 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-cb39fa79-abef-48e6-8d56-96091db7890b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ffa55527bead4570a6f38283af30e012', 'request_id': 'req-cb39fa79-abef-48e6-8d56-96091db7890b'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:37:25.209540 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-cb39fa79-abef-48e6-8d56-96091db7890b admin admin] The token cache is being invalidate because project ffa55527bead4570a6f38283af30e012 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 02:37:25.209832 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-cb39fa79-abef-48e6-8d56-96091db7890b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:37:25.210503 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-cb39fa79-abef-48e6-8d56-96091db7890b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:37:25.212316 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1097/2220] 10.222.0.66 () {58 vars in 1232 bytes} [Mon Jul 18 02:37:24 2022] DELETE /identity/v3/projects/ffa55527bead4570a6f38283af30e012 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:37:25.217341 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518978ed-84ec-4c3f-a2df-9cb6a96f9ff5 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:37:25.217826 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518978ed-84ec-4c3f-a2df-9cb6a96f9ff5 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:37:25.217937 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518978ed-84ec-4c3f-a2df-9cb6a96f9ff5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:37:25.237073 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-518978ed-84ec-4c3f-a2df-9cb6a96f9ff5 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:37:25.238324 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-518978ed-84ec-4c3f-a2df-9cb6a96f9ff5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:37:25.257333 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:37:25.308577 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1124/2221] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:37:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:37:25.317042 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae94b427-0ace-4302-8e58-b344be2c0011 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:37:25.317612 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae94b427-0ace-4302-8e58-b344be2c0011 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:37:25.317612 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae94b427-0ace-4302-8e58-b344be2c0011 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:37:25.336676 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-ae94b427-0ace-4302-8e58-b344be2c0011 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:37:25.337928 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-ae94b427-0ace-4302-8e58-b344be2c0011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:37:25.356822 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1098/2222] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:37:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:37:25.358287 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518978ed-84ec-4c3f-a2df-9cb6a96f9ff5 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:37:25.369022 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518978ed-84ec-4c3f-a2df-9cb6a96f9ff5 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:37:25.373731 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943d8f40-b5d7-41f9-8987-dd6087e1d8ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:37:25.374425 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943d8f40-b5d7-41f9-8987-dd6087e1d8ff admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:37:25.374715 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943d8f40-b5d7-41f9-8987-dd6087e1d8ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:37:25.374994 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943d8f40-b5d7-41f9-8987-dd6087e1d8ff admin admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:37:25.375726 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943d8f40-b5d7-41f9-8987-dd6087e1d8ff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:37:25.376035 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943d8f40-b5d7-41f9-8987-dd6087e1d8ff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:37:25.377766 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943d8f40-b5d7-41f9-8987-dd6087e1d8ff admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:37:25.389490 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1125/2223] 10.222.0.66 () {58 vars in 1152 bytes} [Mon Jul 18 02:37:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:37:25.394656 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae94b427-0ace-4302-8e58-b344be2c0011 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:37:25.404143 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae94b427-0ace-4302-8e58-b344be2c0011 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:37:25.409762 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74872466-25a9-4a38-ab7c-f41facf6dba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:37:25.410440 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74872466-25a9-4a38-ab7c-f41facf6dba7 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:37:25.410723 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74872466-25a9-4a38-ab7c-f41facf6dba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:37:25.411059 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74872466-25a9-4a38-ab7c-f41facf6dba7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:37:25.411747 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74872466-25a9-4a38-ab7c-f41facf6dba7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:37:25.413653 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74872466-25a9-4a38-ab7c-f41facf6dba7 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:37:25.509188 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:37:25.519204 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1099/2224] 10.222.0.66 () {60 vars in 1152 bytes} [Mon Jul 18 02:37:25 2022] POST /identity/v3/projects => generated 379 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:37:25.520814 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943d8f40-b5d7-41f9-8987-dd6087e1d8ff admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:37:25.531029 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943d8f40-b5d7-41f9-8987-dd6087e1d8ff admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:37:25.605308 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906fb12c-02e4-4f19-a9b7-2f0915a1e9d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:37:25.606015 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906fb12c-02e4-4f19-a9b7-2f0915a1e9d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:37:25.606349 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906fb12c-02e4-4f19-a9b7-2f0915a1e9d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:37:25.606638 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906fb12c-02e4-4f19-a9b7-2f0915a1e9d1 admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:37:25.607360 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906fb12c-02e4-4f19-a9b7-2f0915a1e9d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:37:25.609302 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906fb12c-02e4-4f19-a9b7-2f0915a1e9d1 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:37:25.732982 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1126/2225] 10.222.0.66 () {60 vars in 1143 bytes} [Mon Jul 18 02:37:25 2022] POST /identity/v3/users => generated 290 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:37:25.737029 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74872466-25a9-4a38-ab7c-f41facf6dba7 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:37:25.747570 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74872466-25a9-4a38-ab7c-f41facf6dba7 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:37:25.752713 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46595822-014d-4f8f-b841-6788f38ba6cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:37:25.753374 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46595822-014d-4f8f-b841-6788f38ba6cc admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:37:25.753663 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46595822-014d-4f8f-b841-6788f38ba6cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:37:25.754139 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46595822-014d-4f8f-b841-6788f38ba6cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:37:25.754693 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46595822-014d-4f8f-b841-6788f38ba6cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:37:25.756302 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46595822-014d-4f8f-b841-6788f38ba6cc admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:37:25.783417 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1100/2226] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:37:25 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 02:37:25.785579 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906fb12c-02e4-4f19-a9b7-2f0915a1e9d1 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:37:25.807428 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906fb12c-02e4-4f19-a9b7-2f0915a1e9d1 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:37:25.818268 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb82daa-63a1-49c9-be00-13ff63c7878e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:37:25.819023 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb82daa-63a1-49c9-be00-13ff63c7878e admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:37:25.821242 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb82daa-63a1-49c9-be00-13ff63c7878e admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:37:25.821242 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb82daa-63a1-49c9-be00-13ff63c7878e admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:37:25.821242 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb82daa-63a1-49c9-be00-13ff63c7878e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:37:25.821957 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb82daa-63a1-49c9-be00-13ff63c7878e admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:37:25.833376 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1127/2227] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:37:25 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:37:25.837613 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46595822-014d-4f8f-b841-6788f38ba6cc admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:37:25.848210 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46595822-014d-4f8f-b841-6788f38ba6cc admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:37:25.855546 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f1f5eb-2580-457b-9c0e-2e299843fba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:37:25.855546 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f1f5eb-2580-457b-9c0e-2e299843fba5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:37:25.855546 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f1f5eb-2580-457b-9c0e-2e299843fba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:37:25.855546 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f1f5eb-2580-457b-9c0e-2e299843fba5 admin admin] PATH_INFO: `/v3/projects/807f88779cf842f3bbe09615d470f44c/users/41a452eca3af46b09f56a80fa8652247/roles/65feabf3b27944a8b01a86c8ac608457` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:37:25.869891 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f1f5eb-2580-457b-9c0e-2e299843fba5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=807f88779cf842f3bbe09615d470f44c, user_id=41a452eca3af46b09f56a80fa8652247, role_id=65feabf3b27944a8b01a86c8ac608457)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:37:25.872627 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f1f5eb-2580-457b-9c0e-2e299843fba5 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:37:26.055570 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1101/2228] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:37:25 2022] PUT /identity/v3/projects/807f88779cf842f3bbe09615d470f44c/users/41a452eca3af46b09f56a80fa8652247/roles/65feabf3b27944a8b01a86c8ac608457 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:37:26.061468 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd7dc9e-bf50-495f-b892-9f81345d4cfa None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:37:26.062118 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd7dc9e-bf50-495f-b892-9f81345d4cfa None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:37:26.062609 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd7dc9e-bf50-495f-b892-9f81345d4cfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:37:26.084797 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-8fd7dc9e-bf50-495f-b892-9f81345d4cfa None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:37:26.086061 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-8fd7dc9e-bf50-495f-b892-9f81345d4cfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:37:26.150659 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1128/2229] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:37:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:37:26.164766 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f1f5eb-2580-457b-9c0e-2e299843fba5 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:37:26.166766 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-25f1f5eb-2580-457b-9c0e-2e299843fba5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:37:26.225309 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f1f5eb-2580-457b-9c0e-2e299843fba5 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:37:26.229338 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1d8dc9-2711-4b89-90ac-99b86b7a8e39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eaf19a259c2741cea375a38763e33cdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:37:26.230267 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1d8dc9-2711-4b89-90ac-99b86b7a8e39 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:37:26.230798 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1d8dc9-2711-4b89-90ac-99b86b7a8e39 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:37:26.231102 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1d8dc9-2711-4b89-90ac-99b86b7a8e39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:37:26.241076 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1d8dc9-2711-4b89-90ac-99b86b7a8e39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:37:26.254114 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1d8dc9-2711-4b89-90ac-99b86b7a8e39 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:37:26.254114 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1102/2230] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:37:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:37:27.597069 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776e2939-d6a0-4a43-9bc2-946fa23bb92a None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:37:27.597593 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776e2939-d6a0-4a43-9bc2-946fa23bb92a None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:37:27.602143 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776e2939-d6a0-4a43-9bc2-946fa23bb92a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:37:27.624421 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-776e2939-d6a0-4a43-9bc2-946fa23bb92a None None] MFA Rules not processed for user `86d9949aaf5749c9b00e1e04171e6f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:37:27.625686 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-776e2939-d6a0-4a43-9bc2-946fa23bb92a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:37:27.695118 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:37:27.744167 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1129/2231] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:37:27 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:37:32.967573 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc85988-02e7-4db0-ba4b-aa58b6311a66 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:37:32.968041 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc85988-02e7-4db0-ba4b-aa58b6311a66 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:37:32.968225 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc85988-02e7-4db0-ba4b-aa58b6311a66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:37:32.998408 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-0fc85988-02e7-4db0-ba4b-aa58b6311a66 None None] MFA Rules not processed for user `768f872096584d4093fc616d547aabb6`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:37:33.000356 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-0fc85988-02e7-4db0-ba4b-aa58b6311a66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:37:33.095530 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1103/2232] 10.222.0.66 () {58 vars in 1009 bytes} [Mon Jul 18 02:37:32 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:37:33.140383 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776e2939-d6a0-4a43-9bc2-946fa23bb92a None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:37:33.142758 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-776e2939-d6a0-4a43-9bc2-946fa23bb92a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:37:33.304734 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776e2939-d6a0-4a43-9bc2-946fa23bb92a None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:37:33.310195 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e956c2f3-a724-4369-a3e6-d3e54954b2a2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4300622cbd347dd8f5227bb960e98d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:37:33.311448 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e956c2f3-a724-4369-a3e6-d3e54954b2a2 service swift] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:37:33.312337 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e956c2f3-a724-4369-a3e6-d3e54954b2a2 service swift] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:37:33.312629 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e956c2f3-a724-4369-a3e6-d3e54954b2a2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:37:33.322885 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e956c2f3-a724-4369-a3e6-d3e54954b2a2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:37:33.324861 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e956c2f3-a724-4369-a3e6-d3e54954b2a2 service swift] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:37:33.336656 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1130/2233] 10.222.0.66 () {60 vars in 1384 bytes} [Mon Jul 18 02:37:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 02:37:37.856840 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fba4c52-632d-4dcf-b99c-a0f32f0c0ffc None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:37:37.858030 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fba4c52-632d-4dcf-b99c-a0f32f0c0ffc None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:37:37.859095 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fba4c52-632d-4dcf-b99c-a0f32f0c0ffc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:37:37.945722 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-9fba4c52-632d-4dcf-b99c-a0f32f0c0ffc None None] MFA Rules not processed for user `41a452eca3af46b09f56a80fa8652247`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:37:37.950358 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-9fba4c52-632d-4dcf-b99c-a0f32f0c0ffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:37:38.094420 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1104/2234] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:37:37 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:37:38.099622 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e956c2f3-a724-4369-a3e6-d3e54954b2a2 service swift] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:37:38.111855 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e956c2f3-a724-4369-a3e6-d3e54954b2a2 service swift] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:37:38.124108 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c6d015-ab1c-4c7b-9273-64fc123a8333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:37:38.125766 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c6d015-ab1c-4c7b-9273-64fc123a8333 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:37:38.127131 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c6d015-ab1c-4c7b-9273-64fc123a8333 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:37:38.127651 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c6d015-ab1c-4c7b-9273-64fc123a8333 admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:37:38.129872 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c6d015-ab1c-4c7b-9273-64fc123a8333 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:37:38.133664 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c6d015-ab1c-4c7b-9273-64fc123a8333 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:37:38.260321 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1131/2235] 10.222.0.66 () {60 vars in 1152 bytes} [Mon Jul 18 02:37:38 2022] POST /identity/v3/projects => generated 379 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:37:38.264107 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fba4c52-632d-4dcf-b99c-a0f32f0c0ffc None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:37:38.270404 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:37:38.299887 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fba4c52-632d-4dcf-b99c-a0f32f0c0ffc None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:37:38.423285 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2de5a5-bd95-4153-b02f-52784e5677a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:37:38.423762 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2de5a5-bd95-4153-b02f-52784e5677a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:37:38.424045 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2de5a5-bd95-4153-b02f-52784e5677a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:37:38.424319 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2de5a5-bd95-4153-b02f-52784e5677a8 admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:37:38.425018 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2de5a5-bd95-4153-b02f-52784e5677a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:37:38.427844 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2de5a5-bd95-4153-b02f-52784e5677a8 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:37:38.560733 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1105/2236] 10.222.0.66 () {60 vars in 1143 bytes} [Mon Jul 18 02:37:38 2022] POST /identity/v3/users => generated 296 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:37:38.564407 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c6d015-ab1c-4c7b-9273-64fc123a8333 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:37:38.580599 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c6d015-ab1c-4c7b-9273-64fc123a8333 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:37:38.593951 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bc5688-d532-4a36-93e4-2832d5634ea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:37:38.596025 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bc5688-d532-4a36-93e4-2832d5634ea6 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:37:38.600689 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bc5688-d532-4a36-93e4-2832d5634ea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:37:38.601466 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bc5688-d532-4a36-93e4-2832d5634ea6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:37:38.616561 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bc5688-d532-4a36-93e4-2832d5634ea6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:37:38.618418 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bc5688-d532-4a36-93e4-2832d5634ea6 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:37:38.684791 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1132/2237] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:37:38 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:37:38.686135 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2de5a5-bd95-4153-b02f-52784e5677a8 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:37:38.704153 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2de5a5-bd95-4153-b02f-52784e5677a8 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:37:38.721894 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8631d0-0b67-47d6-b6c1-4ea749797234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:37:38.723666 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8631d0-0b67-47d6-b6c1-4ea749797234 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:37:38.724201 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8631d0-0b67-47d6-b6c1-4ea749797234 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:37:38.724704 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8631d0-0b67-47d6-b6c1-4ea749797234 admin admin] PATH_INFO: `/v3/projects/d447d08d2f9c48caaed6d78c2dc46e2b/users/ce8b774659014eccb994e023afca7b9f/roles/d4aa3324f0c84887b41d0dc4acf7f30e` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:37:38.751412 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8631d0-0b67-47d6-b6c1-4ea749797234 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d447d08d2f9c48caaed6d78c2dc46e2b, user_id=ce8b774659014eccb994e023afca7b9f, role_id=d4aa3324f0c84887b41d0dc4acf7f30e)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:37:38.756530 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8631d0-0b67-47d6-b6c1-4ea749797234 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:37:38.844311 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1106/2238] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:37:38 2022] PUT /identity/v3/projects/d447d08d2f9c48caaed6d78c2dc46e2b/users/ce8b774659014eccb994e023afca7b9f/roles/d4aa3324f0c84887b41d0dc4acf7f30e => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:37:38.849041 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bc5688-d532-4a36-93e4-2832d5634ea6 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:37:38.861448 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bc5688-d532-4a36-93e4-2832d5634ea6 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:37:38.970760 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e8f0b3-e94b-4ed2-a670-b41900533cb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:37:38.973067 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e8f0b3-e94b-4ed2-a670-b41900533cb3 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:37:38.973637 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e8f0b3-e94b-4ed2-a670-b41900533cb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:37:38.974022 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e8f0b3-e94b-4ed2-a670-b41900533cb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:37:38.974906 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e8f0b3-e94b-4ed2-a670-b41900533cb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:37:38.976661 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e8f0b3-e94b-4ed2-a670-b41900533cb3 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:37:38.995051 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1133/2239] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:37:38 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:37:39.001332 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8631d0-0b67-47d6-b6c1-4ea749797234 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:37:39.034234 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8631d0-0b67-47d6-b6c1-4ea749797234 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:37:39.049179 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166b32b8-f442-4c22-a30a-4f83ad401437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:37:39.050124 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166b32b8-f442-4c22-a30a-4f83ad401437 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:37:39.051157 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166b32b8-f442-4c22-a30a-4f83ad401437 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:37:39.051447 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166b32b8-f442-4c22-a30a-4f83ad401437 admin admin] PATH_INFO: `/v3/projects/d447d08d2f9c48caaed6d78c2dc46e2b/users/ce8b774659014eccb994e023afca7b9f/roles/d4aa3324f0c84887b41d0dc4acf7f30e` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:37:39.056481 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166b32b8-f442-4c22-a30a-4f83ad401437 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d447d08d2f9c48caaed6d78c2dc46e2b, user_id=ce8b774659014eccb994e023afca7b9f, role_id=d4aa3324f0c84887b41d0dc4acf7f30e)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:37:39.069549 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166b32b8-f442-4c22-a30a-4f83ad401437 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:37:39.094602 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1107/2240] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:37:38 2022] PUT /identity/v3/projects/d447d08d2f9c48caaed6d78c2dc46e2b/users/ce8b774659014eccb994e023afca7b9f/roles/d4aa3324f0c84887b41d0dc4acf7f30e => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:37:48.983597 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=87,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:37:49.087859 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=43,keystone:INSERT=5 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:37:50.149333 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e8f0b3-e94b-4ed2-a670-b41900533cb3 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:37:50.151359 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-a1e8f0b3-e94b-4ed2-a670-b41900533cb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:37:50.171864 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e8f0b3-e94b-4ed2-a670-b41900533cb3 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:37:50.321978 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef77428-9c4a-49c3-ad2f-ce898cea7677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:37:50.323064 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef77428-9c4a-49c3-ad2f-ce898cea7677 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:37:50.323610 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef77428-9c4a-49c3-ad2f-ce898cea7677 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:37:50.323900 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef77428-9c4a-49c3-ad2f-ce898cea7677 admin admin] PATH_INFO: `/v3/users/994a19636bb348ce9fdb015391dca6f1` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:37:50.325202 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef77428-9c4a-49c3-ad2f-ce898cea7677 admin admin] RBAC: Authorizing `identity:delete_user(user_id=994a19636bb348ce9fdb015391dca6f1)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:37:50.327243 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef77428-9c4a-49c3-ad2f-ce898cea7677 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:37:50.683680 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-aef77428-9c4a-49c3-ad2f-ce898cea7677 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '994a19636bb348ce9fdb015391dca6f1'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:37:50.816794 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-aef77428-9c4a-49c3-ad2f-ce898cea7677 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '994a19636bb348ce9fdb015391dca6f1'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:37:50.839214 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-aef77428-9c4a-49c3-ad2f-ce898cea7677 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '994a19636bb348ce9fdb015391dca6f1'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:37:50.841684 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-aef77428-9c4a-49c3-ad2f-ce898cea7677 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '994a19636bb348ce9fdb015391dca6f1'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:37:50.883808 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-aef77428-9c4a-49c3-ad2f-ce898cea7677 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '994a19636bb348ce9fdb015391dca6f1'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:37:50.896497 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1134/2241] 10.222.0.66 () {58 vars in 1223 bytes} [Mon Jul 18 02:37:50 2022] DELETE /identity/v3/users/994a19636bb348ce9fdb015391dca6f1 => generated 0 bytes in 748 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:37:50.901056 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb52f38b-53d4-4083-8f5c-a7de9a34e6b4 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:37:50.901614 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb52f38b-53d4-4083-8f5c-a7de9a34e6b4 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:37:50.901614 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb52f38b-53d4-4083-8f5c-a7de9a34e6b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:37:50.933938 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-fb52f38b-53d4-4083-8f5c-a7de9a34e6b4 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:37:50.936912 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-fb52f38b-53d4-4083-8f5c-a7de9a34e6b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:37:51.060793 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1108/2242] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:37:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:37:51.081503 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef77428-9c4a-49c3-ad2f-ce898cea7677 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:37:51.083514 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-aef77428-9c4a-49c3-ad2f-ce898cea7677 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:37:51.231401 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef77428-9c4a-49c3-ad2f-ce898cea7677 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:37:51.235932 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a532c241-a7d9-445e-8925-83454b44425b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eaf19a259c2741cea375a38763e33cdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:37:51.237080 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a532c241-a7d9-445e-8925-83454b44425b service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:37:51.237574 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a532c241-a7d9-445e-8925-83454b44425b service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:37:51.238048 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a532c241-a7d9-445e-8925-83454b44425b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:37:51.252387 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a532c241-a7d9-445e-8925-83454b44425b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:37:51.254312 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a532c241-a7d9-445e-8925-83454b44425b service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:37:51.266209 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1135/2243] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:37:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 02:37:51.943903 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e609238-eaa9-44f3-96e2-098eb8f11acc None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:37:51.944897 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e609238-eaa9-44f3-96e2-098eb8f11acc None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:37:51.945333 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e609238-eaa9-44f3-96e2-098eb8f11acc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:37:52.004384 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-7e609238-eaa9-44f3-96e2-098eb8f11acc None None] MFA Rules not processed for user `ce8b774659014eccb994e023afca7b9f`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:37:52.013754 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-7e609238-eaa9-44f3-96e2-098eb8f11acc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:37:52.141918 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1109/2244] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:37:51 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:37:52.256737 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a532c241-a7d9-445e-8925-83454b44425b service neutron] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:37:52.258210 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-a532c241-a7d9-445e-8925-83454b44425b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:37:52.293534 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:37:52.351424 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a532c241-a7d9-445e-8925-83454b44425b service neutron] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:37:52.358358 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ec3170-ad1e-4c9d-b49c-4d53516faf74 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80b138831bc54c7b986d50092d62d4af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:37:52.359161 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ec3170-ad1e-4c9d-b49c-4d53516faf74 service nova] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:37:52.364041 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ec3170-ad1e-4c9d-b49c-4d53516faf74 service nova] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:37:52.364041 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ec3170-ad1e-4c9d-b49c-4d53516faf74 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:37:52.364041 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-59ec3170-ad1e-4c9d-b49c-4d53516faf74 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:37:52.436219 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ec3170-ad1e-4c9d-b49c-4d53516faf74 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:37:52.437849 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ec3170-ad1e-4c9d-b49c-4d53516faf74 service nova] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:37:52.449068 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1136/2245] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:37:52 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:37:52.682416 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e609238-eaa9-44f3-96e2-098eb8f11acc None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:37:52.682416 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-7e609238-eaa9-44f3-96e2-098eb8f11acc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:37:52.693057 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e609238-eaa9-44f3-96e2-098eb8f11acc None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:37:52.697705 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dc95e5-1df1-407f-accf-6d5c483f0344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:37:52.698617 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dc95e5-1df1-407f-accf-6d5c483f0344 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:37:52.698908 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dc95e5-1df1-407f-accf-6d5c483f0344 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:37:52.699189 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dc95e5-1df1-407f-accf-6d5c483f0344 admin admin] PATH_INFO: `/v3/projects/2f35aa2ec35f41448e63643c5c043ea8` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:37:52.700395 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dc95e5-1df1-407f-accf-6d5c483f0344 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f35aa2ec35f41448e63643c5c043ea8)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:37:52.702381 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dc95e5-1df1-407f-accf-6d5c483f0344 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:37:52.854970 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-52dc95e5-1df1-407f-accf-6d5c483f0344 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f35aa2ec35f41448e63643c5c043ea8', 'request_id': 'req-52dc95e5-1df1-407f-accf-6d5c483f0344'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:37:52.869908 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-52dc95e5-1df1-407f-accf-6d5c483f0344 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f35aa2ec35f41448e63643c5c043ea8', 'request_id': 'req-52dc95e5-1df1-407f-accf-6d5c483f0344'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:37:52.878654 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-52dc95e5-1df1-407f-accf-6d5c483f0344 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f35aa2ec35f41448e63643c5c043ea8', 'request_id': 'req-52dc95e5-1df1-407f-accf-6d5c483f0344'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:37:53.008937 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-52dc95e5-1df1-407f-accf-6d5c483f0344 admin admin] The token cache is being invalidate because project 2f35aa2ec35f41448e63643c5c043ea8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 02:37:53.008937 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-52dc95e5-1df1-407f-accf-6d5c483f0344 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:37:53.008937 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-52dc95e5-1df1-407f-accf-6d5c483f0344 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:37:53.008937 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1110/2246] 10.222.0.66 () {58 vars in 1232 bytes} [Mon Jul 18 02:37:52 2022] DELETE /identity/v3/projects/2f35aa2ec35f41448e63643c5c043ea8 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:37:53.013810 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38af6380-5eae-4156-aaab-cdd472d8289e None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:37:53.014119 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38af6380-5eae-4156-aaab-cdd472d8289e None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:37:53.014601 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38af6380-5eae-4156-aaab-cdd472d8289e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:37:53.054421 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-38af6380-5eae-4156-aaab-cdd472d8289e None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:37:53.055671 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-38af6380-5eae-4156-aaab-cdd472d8289e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:37:53.167493 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1137/2247] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:37:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:37:53.175956 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589b0c63-1715-4edc-9511-3b173a375934 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:37:53.176558 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589b0c63-1715-4edc-9511-3b173a375934 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:37:53.177006 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589b0c63-1715-4edc-9511-3b173a375934 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:37:53.225240 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-589b0c63-1715-4edc-9511-3b173a375934 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:37:53.226687 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-589b0c63-1715-4edc-9511-3b173a375934 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:37:53.236027 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1111/2248] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:37:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:37:53.240560 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38af6380-5eae-4156-aaab-cdd472d8289e None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:37:53.252965 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38af6380-5eae-4156-aaab-cdd472d8289e None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:37:53.265980 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cca861-f370-429a-b73a-5ab8f79fd08e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:37:53.267696 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cca861-f370-429a-b73a-5ab8f79fd08e admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:37:53.268235 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cca861-f370-429a-b73a-5ab8f79fd08e admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:37:53.268671 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cca861-f370-429a-b73a-5ab8f79fd08e admin admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:37:53.269591 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cca861-f370-429a-b73a-5ab8f79fd08e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:37:53.270045 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cca861-f370-429a-b73a-5ab8f79fd08e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:37:53.271906 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cca861-f370-429a-b73a-5ab8f79fd08e admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:37:53.285245 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1138/2249] 10.222.0.66 () {58 vars in 1152 bytes} [Mon Jul 18 02:37:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:37:53.289770 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589b0c63-1715-4edc-9511-3b173a375934 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:37:53.295060 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:37:53.303908 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589b0c63-1715-4edc-9511-3b173a375934 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:37:53.312518 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19bb5c1-d1c5-4322-a40d-975767a9be9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:37:53.312518 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19bb5c1-d1c5-4322-a40d-975767a9be9d admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:37:53.312518 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19bb5c1-d1c5-4322-a40d-975767a9be9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:37:53.312518 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19bb5c1-d1c5-4322-a40d-975767a9be9d admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:37:53.312943 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19bb5c1-d1c5-4322-a40d-975767a9be9d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:37:53.314691 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19bb5c1-d1c5-4322-a40d-975767a9be9d admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:37:53.436614 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1112/2250] 10.222.0.66 () {60 vars in 1152 bytes} [Mon Jul 18 02:37:53 2022] POST /identity/v3/projects => generated 375 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:37:53.440760 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cca861-f370-429a-b73a-5ab8f79fd08e admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:37:53.455547 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cca861-f370-429a-b73a-5ab8f79fd08e admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:37:53.577297 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:37:53.618060 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc9508f-d20d-4408-b176-183904fc12a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:37:53.618913 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc9508f-d20d-4408-b176-183904fc12a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:37:53.619716 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc9508f-d20d-4408-b176-183904fc12a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:37:53.620008 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc9508f-d20d-4408-b176-183904fc12a7 admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:37:53.620731 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc9508f-d20d-4408-b176-183904fc12a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:37:53.622734 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc9508f-d20d-4408-b176-183904fc12a7 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:37:53.775347 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1139/2251] 10.222.0.66 () {60 vars in 1143 bytes} [Mon Jul 18 02:37:53 2022] POST /identity/v3/users => generated 288 bytes in 335 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:37:53.775640 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19bb5c1-d1c5-4322-a40d-975767a9be9d admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:37:53.797367 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19bb5c1-d1c5-4322-a40d-975767a9be9d admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:37:53.810695 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0348681c-7dbc-4035-a1ba-38669456ef0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:37:53.811404 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0348681c-7dbc-4035-a1ba-38669456ef0e admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:37:53.811762 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0348681c-7dbc-4035-a1ba-38669456ef0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:37:53.812046 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0348681c-7dbc-4035-a1ba-38669456ef0e admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:37:53.812867 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0348681c-7dbc-4035-a1ba-38669456ef0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:37:53.818420 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0348681c-7dbc-4035-a1ba-38669456ef0e admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:37:53.874814 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1113/2252] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:37:53 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:37:53.876551 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc9508f-d20d-4408-b176-183904fc12a7 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:37:53.928803 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc9508f-d20d-4408-b176-183904fc12a7 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:37:53.928803 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e726d42-44a3-4670-8681-c4f403e78da8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:37:53.928803 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e726d42-44a3-4670-8681-c4f403e78da8 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:37:53.928803 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e726d42-44a3-4670-8681-c4f403e78da8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:37:53.929233 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e726d42-44a3-4670-8681-c4f403e78da8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:37:53.929233 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e726d42-44a3-4670-8681-c4f403e78da8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:37:53.929233 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e726d42-44a3-4670-8681-c4f403e78da8 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:37:53.929233 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1140/2253] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:37:53 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:37:53.935068 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0348681c-7dbc-4035-a1ba-38669456ef0e admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:37:53.946438 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0348681c-7dbc-4035-a1ba-38669456ef0e admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:37:53.951660 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c3ff3b-9553-4aa3-9349-f5cd02db2cac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:37:53.952653 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c3ff3b-9553-4aa3-9349-f5cd02db2cac admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:37:53.953088 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c3ff3b-9553-4aa3-9349-f5cd02db2cac admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:37:53.953455 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c3ff3b-9553-4aa3-9349-f5cd02db2cac admin admin] PATH_INFO: `/v3/projects/ba114b0e00424785a219aa31d987d2ee/users/273664a60a334dc58ede22f9ccc0a128/roles/65feabf3b27944a8b01a86c8ac608457` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:37:53.971888 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c3ff3b-9553-4aa3-9349-f5cd02db2cac admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba114b0e00424785a219aa31d987d2ee, user_id=273664a60a334dc58ede22f9ccc0a128, role_id=65feabf3b27944a8b01a86c8ac608457)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:37:53.976415 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c3ff3b-9553-4aa3-9349-f5cd02db2cac admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:37:54.216639 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1114/2254] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:37:53 2022] PUT /identity/v3/projects/ba114b0e00424785a219aa31d987d2ee/users/273664a60a334dc58ede22f9ccc0a128/roles/65feabf3b27944a8b01a86c8ac608457 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:37:54.293470 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7552499e-f232-4d4e-aa82-c645d97df0bc None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:37:54.293470 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7552499e-f232-4d4e-aa82-c645d97df0bc None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:37:54.298063 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7552499e-f232-4d4e-aa82-c645d97df0bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:37:54.337675 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-7552499e-f232-4d4e-aa82-c645d97df0bc None None] MFA Rules not processed for user `273664a60a334dc58ede22f9ccc0a128`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:37:54.338977 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-7552499e-f232-4d4e-aa82-c645d97df0bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:37:54.459187 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c3ff3b-9553-4aa3-9349-f5cd02db2cac admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:37:54.460375 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1141/2255] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:37:54 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:37:54.461214 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-c8c3ff3b-9553-4aa3-9349-f5cd02db2cac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:37:54.545235 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c3ff3b-9553-4aa3-9349-f5cd02db2cac admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:37:54.548957 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243b276f-0bfa-4533-bc8e-c5d1953a99b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd49388facdf446885e55b45ef91f01e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:37:54.549614 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243b276f-0bfa-4533-bc8e-c5d1953a99b3 service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:37:54.549898 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243b276f-0bfa-4533-bc8e-c5d1953a99b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:37:54.550202 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243b276f-0bfa-4533-bc8e-c5d1953a99b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:37:54.561352 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243b276f-0bfa-4533-bc8e-c5d1953a99b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:37:54.563248 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243b276f-0bfa-4533-bc8e-c5d1953a99b3 service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:37:54.575273 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1115/2256] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:37:54 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 02:37:57.812401 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9275a38b-9e2d-4847-b4fd-6e5328c1fcc3 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:37:57.813326 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9275a38b-9e2d-4847-b4fd-6e5328c1fcc3 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:37:57.813770 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9275a38b-9e2d-4847-b4fd-6e5328c1fcc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:37:57.834825 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-9275a38b-9e2d-4847-b4fd-6e5328c1fcc3 None None] MFA Rules not processed for user `273664a60a334dc58ede22f9ccc0a128`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:37:57.836221 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-9275a38b-9e2d-4847-b4fd-6e5328c1fcc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:37:57.851813 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1142/2257] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:37:57 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:38:01.298810 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a754a8-cd99-49f2-8044-c579c507e25b None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:38:01.299307 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a754a8-cd99-49f2-8044-c579c507e25b None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:38:01.299421 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a754a8-cd99-49f2-8044-c579c507e25b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:38:01.317150 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-60a754a8-cd99-49f2-8044-c579c507e25b None None] MFA Rules not processed for user `d281c92e20d04004a679a8ec65f9e6d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:38:01.318449 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-60a754a8-cd99-49f2-8044-c579c507e25b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:38:01.376811 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1116/2258] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:38:01 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:38:03.575736 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1504f25b-d95c-46ac-a960-9949a15903b5 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:38:03.576333 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1504f25b-d95c-46ac-a960-9949a15903b5 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:38:03.576333 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1504f25b-d95c-46ac-a960-9949a15903b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:38:03.597273 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-1504f25b-d95c-46ac-a960-9949a15903b5 None None] MFA Rules not processed for user `768f872096584d4093fc616d547aabb6`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:38:03.598511 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-1504f25b-d95c-46ac-a960-9949a15903b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:38:03.656789 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1143/2259] 10.222.0.66 () {58 vars in 1009 bytes} [Mon Jul 18 02:38:03 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:38:03.670469 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a754a8-cd99-49f2-8044-c579c507e25b None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:38:03.672033 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-60a754a8-cd99-49f2-8044-c579c507e25b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:38:03.739051 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:38:03.770068 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a754a8-cd99-49f2-8044-c579c507e25b None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:38:03.775600 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7374ee-1a31-40d2-88f1-59bad2413ad8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4300622cbd347dd8f5227bb960e98d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:38:03.776254 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7374ee-1a31-40d2-88f1-59bad2413ad8 service swift] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:38:03.776533 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7374ee-1a31-40d2-88f1-59bad2413ad8 service swift] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:38:03.776805 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7374ee-1a31-40d2-88f1-59bad2413ad8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:38:03.785682 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7374ee-1a31-40d2-88f1-59bad2413ad8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:38:03.787312 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7374ee-1a31-40d2-88f1-59bad2413ad8 service swift] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:38:03.796066 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1117/2260] 10.222.0.66 () {60 vars in 1384 bytes} [Mon Jul 18 02:38:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 02:38:04.120550 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562f5d53-70c6-43a8-a36a-8a38068cb3e9 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:38:04.121096 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562f5d53-70c6-43a8-a36a-8a38068cb3e9 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:38:04.121530 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562f5d53-70c6-43a8-a36a-8a38068cb3e9 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:38:04.123193 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1144/2261] 10.222.0.66 () {54 vars in 921 bytes} [Mon Jul 18 02:38:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:38:04.132583 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5eeb4a-7021-44c3-9c0e-090e9f74c2f2 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:38:04.133361 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5eeb4a-7021-44c3-9c0e-090e9f74c2f2 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:38:04.133971 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5eeb4a-7021-44c3-9c0e-090e9f74c2f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:38:04.161865 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-2c5eeb4a-7021-44c3-9c0e-090e9f74c2f2 None None] MFA Rules not processed for user `80b138831bc54c7b986d50092d62d4af`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:38:04.163822 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-2c5eeb4a-7021-44c3-9c0e-090e9f74c2f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:38:04.326191 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1118/2262] 10.222.0.66 () {58 vars in 1018 bytes} [Mon Jul 18 02:38:04 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:38:12.968532 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562f5d53-70c6-43a8-a36a-8a38068cb3e9 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:38:12.971073 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-562f5d53-70c6-43a8-a36a-8a38068cb3e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:38:13.038306 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562f5d53-70c6-43a8-a36a-8a38068cb3e9 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:38:13.042847 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66d4520-e5fe-412c-831a-e68bab670901 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1391b81e7234bc18e22b9e84f231256', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:38:13.043502 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66d4520-e5fe-412c-831a-e68bab670901 service glance] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:38:13.043790 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66d4520-e5fe-412c-831a-e68bab670901 service glance] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:38:13.044072 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66d4520-e5fe-412c-831a-e68bab670901 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:38:13.052692 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:38:13.055827 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66d4520-e5fe-412c-831a-e68bab670901 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:38:13.057238 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66d4520-e5fe-412c-831a-e68bab670901 service glance] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:38:13.071766 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1145/2263] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:38:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:38:13.400013 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b154ba-c4bf-4548-abd9-ce94d01b3aa3 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:38:13.400013 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b154ba-c4bf-4548-abd9-ce94d01b3aa3 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:38:13.400013 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b154ba-c4bf-4548-abd9-ce94d01b3aa3 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:38:13.401426 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1119/2264] 10.222.0.66 () {54 vars in 921 bytes} [Mon Jul 18 02:38:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:38:13.411778 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126528d0-6a91-4509-9c00-364ebed80534 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:38:13.412067 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126528d0-6a91-4509-9c00-364ebed80534 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:38:13.412395 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126528d0-6a91-4509-9c00-364ebed80534 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:38:13.442127 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-126528d0-6a91-4509-9c00-364ebed80534 None None] MFA Rules not processed for user `eaf19a259c2741cea375a38763e33cdc`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:38:13.443271 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-126528d0-6a91-4509-9c00-364ebed80534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:38:13.523252 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1146/2265] 10.222.0.66 () {58 vars in 1018 bytes} [Mon Jul 18 02:38:13 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:38:13.535024 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b154ba-c4bf-4548-abd9-ce94d01b3aa3 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:38:13.536710 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-c0b154ba-c4bf-4548-abd9-ce94d01b3aa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:38:13.550315 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b154ba-c4bf-4548-abd9-ce94d01b3aa3 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:38:13.554115 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12e5611-7e49-4919-8cf1-8132b1f04937 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eaf19a259c2741cea375a38763e33cdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:38:13.555313 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12e5611-7e49-4919-8cf1-8132b1f04937 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:38:13.555708 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12e5611-7e49-4919-8cf1-8132b1f04937 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:38:13.555995 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12e5611-7e49-4919-8cf1-8132b1f04937 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:38:13.563507 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12e5611-7e49-4919-8cf1-8132b1f04937 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:38:13.565034 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12e5611-7e49-4919-8cf1-8132b1f04937 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:38:13.573328 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1120/2266] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:38:13 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:38:13.704716 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6fd5fc-2059-4a1c-82a2-4b437dcf0e96 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:38:13.705006 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6fd5fc-2059-4a1c-82a2-4b437dcf0e96 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:38:13.705289 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6fd5fc-2059-4a1c-82a2-4b437dcf0e96 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:38:13.707870 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1147/2267] 10.222.0.66 () {54 vars in 921 bytes} [Mon Jul 18 02:38:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:38:13.714533 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e73d18-4c54-447b-b2ed-8378c4373ecb None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:38:13.714825 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e73d18-4c54-447b-b2ed-8378c4373ecb None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:38:13.715119 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e73d18-4c54-447b-b2ed-8378c4373ecb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:38:13.735186 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-f9e73d18-4c54-447b-b2ed-8378c4373ecb None None] MFA Rules not processed for user `80b138831bc54c7b986d50092d62d4af`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:38:13.736682 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-f9e73d18-4c54-447b-b2ed-8378c4373ecb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:38:13.747426 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1121/2268] 10.222.0.66 () {58 vars in 1018 bytes} [Mon Jul 18 02:38:13 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:38:13.766869 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6fd5fc-2059-4a1c-82a2-4b437dcf0e96 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:38:13.780595 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6fd5fc-2059-4a1c-82a2-4b437dcf0e96 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:38:13.784766 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1de0f40-5a26-40ff-8c5b-671da59b3db7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eaf19a259c2741cea375a38763e33cdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:38:13.788547 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1de0f40-5a26-40ff-8c5b-671da59b3db7 service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:38:13.788547 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1de0f40-5a26-40ff-8c5b-671da59b3db7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:38:13.788547 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1de0f40-5a26-40ff-8c5b-671da59b3db7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:38:13.799896 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1de0f40-5a26-40ff-8c5b-671da59b3db7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:38:13.801670 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1de0f40-5a26-40ff-8c5b-671da59b3db7 service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:38:13.818521 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1148/2269] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:38:13 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:38:18.795208 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e73d18-4c54-447b-b2ed-8378c4373ecb None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:38:18.797767 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-f9e73d18-4c54-447b-b2ed-8378c4373ecb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:38:18.953995 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e73d18-4c54-447b-b2ed-8378c4373ecb None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:38:18.969070 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5662979-0b12-4c1a-a600-7ab4688de1ad service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88936b31709b429f8b6777c673ed7b8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:38:18.975509 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5662979-0b12-4c1a-a600-7ab4688de1ad service placement] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:38:18.975509 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5662979-0b12-4c1a-a600-7ab4688de1ad service placement] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:38:18.975509 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5662979-0b12-4c1a-a600-7ab4688de1ad service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:38:18.975509 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-f5662979-0b12-4c1a-a600-7ab4688de1ad service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:38:18.988967 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5662979-0b12-4c1a-a600-7ab4688de1ad service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:38:18.990751 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5662979-0b12-4c1a-a600-7ab4688de1ad service placement] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:38:19.002318 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1122/2270] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:38:18 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:38:20.489385 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bc44c2-e177-4627-885d-cdcca2e0b586 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:38:20.490075 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bc44c2-e177-4627-885d-cdcca2e0b586 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:38:20.490075 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bc44c2-e177-4627-885d-cdcca2e0b586 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:38:20.520477 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-07bc44c2-e177-4627-885d-cdcca2e0b586 None None] MFA Rules not processed for user `41a452eca3af46b09f56a80fa8652247`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:38:20.522158 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-07bc44c2-e177-4627-885d-cdcca2e0b586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:38:20.658770 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1149/2271] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:38:20 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:38:26.711445 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09add275-bb34-4628-a050-b6a4d3d41da7 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:38:26.715392 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09add275-bb34-4628-a050-b6a4d3d41da7 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:38:26.715392 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09add275-bb34-4628-a050-b6a4d3d41da7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:38:26.736528 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-09add275-bb34-4628-a050-b6a4d3d41da7 None None] MFA Rules not processed for user `768f872096584d4093fc616d547aabb6`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:38:26.738200 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-09add275-bb34-4628-a050-b6a4d3d41da7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:38:26.756002 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1123/2272] 10.222.0.66 () {58 vars in 1009 bytes} [Mon Jul 18 02:38:26 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:38:26.764651 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bc44c2-e177-4627-885d-cdcca2e0b586 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:38:26.785944 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bc44c2-e177-4627-885d-cdcca2e0b586 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:38:26.792019 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89889fd-af60-4bc4-9866-5fc37e02099a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4300622cbd347dd8f5227bb960e98d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:38:26.792929 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89889fd-af60-4bc4-9866-5fc37e02099a service swift] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:38:26.793444 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89889fd-af60-4bc4-9866-5fc37e02099a service swift] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:38:26.793940 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89889fd-af60-4bc4-9866-5fc37e02099a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:38:26.804805 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89889fd-af60-4bc4-9866-5fc37e02099a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:38:26.807568 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89889fd-af60-4bc4-9866-5fc37e02099a service swift] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:38:26.824385 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1150/2273] 10.222.0.66 () {60 vars in 1384 bytes} [Mon Jul 18 02:38:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 02:38:27.596263 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c245ad5-f88c-49d3-97e8-b15f715f5309 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:38:27.596821 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c245ad5-f88c-49d3-97e8-b15f715f5309 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:38:27.597268 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c245ad5-f88c-49d3-97e8-b15f715f5309 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:38:27.625802 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-8c245ad5-f88c-49d3-97e8-b15f715f5309 None None] MFA Rules not processed for user `768f872096584d4093fc616d547aabb6`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:38:27.627631 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-8c245ad5-f88c-49d3-97e8-b15f715f5309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:38:27.670115 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1124/2274] 10.222.0.66 () {58 vars in 967 bytes} [Mon Jul 18 02:38:27 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:38:27.692394 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89889fd-af60-4bc4-9866-5fc37e02099a service swift] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:38:27.708884 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89889fd-af60-4bc4-9866-5fc37e02099a service swift] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:38:27.716258 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6107352-09f0-4e6c-ae76-4252153a7617 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4300622cbd347dd8f5227bb960e98d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:38:27.717164 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6107352-09f0-4e6c-ae76-4252153a7617 service swift] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:38:27.717602 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6107352-09f0-4e6c-ae76-4252153a7617 service swift] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:38:27.718042 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6107352-09f0-4e6c-ae76-4252153a7617 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:38:27.746927 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6107352-09f0-4e6c-ae76-4252153a7617 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:38:27.749601 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6107352-09f0-4e6c-ae76-4252153a7617 service swift] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:38:27.766371 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1151/2275] 10.222.0.66 () {60 vars in 1384 bytes} [Mon Jul 18 02:38:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 02:38:33.148872 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c245ad5-f88c-49d3-97e8-b15f715f5309 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:38:33.150502 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-8c245ad5-f88c-49d3-97e8-b15f715f5309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:38:33.212751 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:38:33.247427 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c245ad5-f88c-49d3-97e8-b15f715f5309 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:38:33.252935 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308e2a3b-733f-4b78-9969-cb96c75cb414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:38:33.254020 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308e2a3b-733f-4b78-9969-cb96c75cb414 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:38:33.254510 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308e2a3b-733f-4b78-9969-cb96c75cb414 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:38:33.254965 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308e2a3b-733f-4b78-9969-cb96c75cb414 admin admin] PATH_INFO: `/v3/users/86d9949aaf5749c9b00e1e04171e6f93` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:38:33.256374 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308e2a3b-733f-4b78-9969-cb96c75cb414 admin admin] RBAC: Authorizing `identity:delete_user(user_id=86d9949aaf5749c9b00e1e04171e6f93)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:38:33.258476 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308e2a3b-733f-4b78-9969-cb96c75cb414 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:38:33.833153 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-308e2a3b-733f-4b78-9969-cb96c75cb414 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86d9949aaf5749c9b00e1e04171e6f93'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:38:33.921546 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-308e2a3b-733f-4b78-9969-cb96c75cb414 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86d9949aaf5749c9b00e1e04171e6f93'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:38:33.922199 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-308e2a3b-733f-4b78-9969-cb96c75cb414 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86d9949aaf5749c9b00e1e04171e6f93'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:38:33.936129 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-308e2a3b-733f-4b78-9969-cb96c75cb414 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86d9949aaf5749c9b00e1e04171e6f93'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:38:33.936925 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-308e2a3b-733f-4b78-9969-cb96c75cb414 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86d9949aaf5749c9b00e1e04171e6f93'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:38:33.958830 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1125/2276] 10.222.0.66 () {58 vars in 1223 bytes} [Mon Jul 18 02:38:33 2022] DELETE /identity/v3/users/86d9949aaf5749c9b00e1e04171e6f93 => generated 0 bytes in 806 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:38:34.295253 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6107352-09f0-4e6c-ae76-4252153a7617 service swift] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:38:34.307157 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6107352-09f0-4e6c-ae76-4252153a7617 service swift] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:38:34.406777 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0a68cd-e36e-4453-831d-3e326a111fa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:38:34.407659 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0a68cd-e36e-4453-831d-3e326a111fa0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:38:34.407947 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0a68cd-e36e-4453-831d-3e326a111fa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:38:34.408256 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0a68cd-e36e-4453-831d-3e326a111fa0 admin admin] PATH_INFO: `/v3/projects/d615520de8a64205981c88430498f5f6` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:38:34.409479 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0a68cd-e36e-4453-831d-3e326a111fa0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d615520de8a64205981c88430498f5f6)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:38:34.411459 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0a68cd-e36e-4453-831d-3e326a111fa0 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:38:34.428636 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:38:34.565789 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-6d0a68cd-e36e-4453-831d-3e326a111fa0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd615520de8a64205981c88430498f5f6', 'request_id': 'req-6d0a68cd-e36e-4453-831d-3e326a111fa0'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:38:34.571543 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-6d0a68cd-e36e-4453-831d-3e326a111fa0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd615520de8a64205981c88430498f5f6', 'request_id': 'req-6d0a68cd-e36e-4453-831d-3e326a111fa0'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:38:34.641935 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-6d0a68cd-e36e-4453-831d-3e326a111fa0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd615520de8a64205981c88430498f5f6', 'request_id': 'req-6d0a68cd-e36e-4453-831d-3e326a111fa0'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:38:34.655207 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-6d0a68cd-e36e-4453-831d-3e326a111fa0 admin admin] The token cache is being invalidate because project d615520de8a64205981c88430498f5f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 02:38:34.655500 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-6d0a68cd-e36e-4453-831d-3e326a111fa0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:38:34.656173 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-6d0a68cd-e36e-4453-831d-3e326a111fa0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:38:34.657780 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1152/2277] 10.222.0.66 () {58 vars in 1232 bytes} [Mon Jul 18 02:38:34 2022] DELETE /identity/v3/projects/d615520de8a64205981c88430498f5f6 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:38:34.662916 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed94a31-ed4a-42a6-9e19-4c04d885fa8f None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:38:34.663462 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed94a31-ed4a-42a6-9e19-4c04d885fa8f None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:38:34.663462 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed94a31-ed4a-42a6-9e19-4c04d885fa8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:38:34.683854 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-8ed94a31-ed4a-42a6-9e19-4c04d885fa8f None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:38:34.685496 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-8ed94a31-ed4a-42a6-9e19-4c04d885fa8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:38:34.756583 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1126/2278] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:38:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 02:38:34.765107 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaced98e-f863-424e-ad7c-084738e6d5df None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:38:34.770642 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaced98e-f863-424e-ad7c-084738e6d5df None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:38:34.770642 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaced98e-f863-424e-ad7c-084738e6d5df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:38:34.786444 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-eaced98e-f863-424e-ad7c-084738e6d5df None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:38:34.788166 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-eaced98e-f863-424e-ad7c-084738e6d5df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:38:34.804433 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1153/2279] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:38:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:38:34.809425 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed94a31-ed4a-42a6-9e19-4c04d885fa8f None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:38:34.819997 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed94a31-ed4a-42a6-9e19-4c04d885fa8f None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:38:34.825293 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a43ee0-99a2-41d0-9018-289202c964fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:38:34.825972 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a43ee0-99a2-41d0-9018-289202c964fd admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:38:34.826302 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a43ee0-99a2-41d0-9018-289202c964fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:38:34.826582 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a43ee0-99a2-41d0-9018-289202c964fd admin admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:38:34.827390 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a43ee0-99a2-41d0-9018-289202c964fd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:38:34.827705 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a43ee0-99a2-41d0-9018-289202c964fd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:38:34.829466 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a43ee0-99a2-41d0-9018-289202c964fd admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:38:34.842868 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1127/2280] 10.222.0.66 () {58 vars in 1152 bytes} [Mon Jul 18 02:38:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:38:34.847264 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaced98e-f863-424e-ad7c-084738e6d5df None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:38:34.856937 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaced98e-f863-424e-ad7c-084738e6d5df None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:38:34.862154 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369acc49-45c2-420e-a31b-1e8f0ff7385c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:38:34.862856 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369acc49-45c2-420e-a31b-1e8f0ff7385c admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:38:34.863180 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369acc49-45c2-420e-a31b-1e8f0ff7385c admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:38:34.864571 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369acc49-45c2-420e-a31b-1e8f0ff7385c admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:38:34.865276 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369acc49-45c2-420e-a31b-1e8f0ff7385c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:38:34.867262 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369acc49-45c2-420e-a31b-1e8f0ff7385c admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:38:35.010307 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1154/2281] 10.222.0.66 () {60 vars in 1152 bytes} [Mon Jul 18 02:38:34 2022] POST /identity/v3/projects => generated 389 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:38:35.014687 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a43ee0-99a2-41d0-9018-289202c964fd admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:38:35.035140 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a43ee0-99a2-41d0-9018-289202c964fd admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:38:35.071060 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:38:35.106969 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7026e27-a71c-422e-a776-b28d96c4484c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:38:35.107945 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7026e27-a71c-422e-a776-b28d96c4484c admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:38:35.108226 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7026e27-a71c-422e-a776-b28d96c4484c admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:38:35.108531 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7026e27-a71c-422e-a776-b28d96c4484c admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:38:35.109239 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7026e27-a71c-422e-a776-b28d96c4484c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:38:35.111243 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7026e27-a71c-422e-a776-b28d96c4484c admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:38:35.222868 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1128/2282] 10.222.0.66 () {60 vars in 1143 bytes} [Mon Jul 18 02:38:35 2022] POST /identity/v3/users => generated 295 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:38:35.225298 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369acc49-45c2-420e-a31b-1e8f0ff7385c admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:38:35.235996 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369acc49-45c2-420e-a31b-1e8f0ff7385c admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:38:35.241703 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7a076a-4477-4bc3-a99e-fddff8f408db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:38:35.242524 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7a076a-4477-4bc3-a99e-fddff8f408db admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:38:35.247425 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7a076a-4477-4bc3-a99e-fddff8f408db admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:38:35.247425 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7a076a-4477-4bc3-a99e-fddff8f408db admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:38:35.247425 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7a076a-4477-4bc3-a99e-fddff8f408db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:38:35.247425 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7a076a-4477-4bc3-a99e-fddff8f408db admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:38:35.262124 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1155/2283] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:38:35 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:38:35.263425 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7026e27-a71c-422e-a776-b28d96c4484c admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:38:35.274363 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7026e27-a71c-422e-a776-b28d96c4484c admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:38:35.279722 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c416c9fe-9ca3-4f07-b2e6-9bf1c373e408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:38:35.280446 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c416c9fe-9ca3-4f07-b2e6-9bf1c373e408 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:38:35.280760 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c416c9fe-9ca3-4f07-b2e6-9bf1c373e408 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:38:35.280979 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c416c9fe-9ca3-4f07-b2e6-9bf1c373e408 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:38:35.281703 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c416c9fe-9ca3-4f07-b2e6-9bf1c373e408 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:38:35.283561 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c416c9fe-9ca3-4f07-b2e6-9bf1c373e408 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:38:35.295780 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1129/2284] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:38:35 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:38:35.300048 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7a076a-4477-4bc3-a99e-fddff8f408db admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:38:35.310784 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7a076a-4477-4bc3-a99e-fddff8f408db admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:38:35.315354 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318c6b68-d4ed-4e09-bfbe-6490c0e76d7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:38:35.316222 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318c6b68-d4ed-4e09-bfbe-6490c0e76d7d admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:38:35.316519 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318c6b68-d4ed-4e09-bfbe-6490c0e76d7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:38:35.316772 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318c6b68-d4ed-4e09-bfbe-6490c0e76d7d admin admin] PATH_INFO: `/v3/projects/4d6bd46b2dcb4bea8c4215742c4e70d3/users/2edc56fb488a46bab30cce31410714de/roles/65feabf3b27944a8b01a86c8ac608457` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:38:35.332006 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318c6b68-d4ed-4e09-bfbe-6490c0e76d7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d6bd46b2dcb4bea8c4215742c4e70d3, user_id=2edc56fb488a46bab30cce31410714de, role_id=65feabf3b27944a8b01a86c8ac608457)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:38:35.334718 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318c6b68-d4ed-4e09-bfbe-6490c0e76d7d admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:38:35.429095 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1156/2285] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:38:35 2022] PUT /identity/v3/projects/4d6bd46b2dcb4bea8c4215742c4e70d3/users/2edc56fb488a46bab30cce31410714de/roles/65feabf3b27944a8b01a86c8ac608457 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 02:38:35.440341 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e220cc2-90db-44c6-a70d-7d7fd2857922 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:38:35.440632 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e220cc2-90db-44c6-a70d-7d7fd2857922 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:38:35.440914 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e220cc2-90db-44c6-a70d-7d7fd2857922 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:38:35.476042 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-5e220cc2-90db-44c6-a70d-7d7fd2857922 None None] MFA Rules not processed for user `2edc56fb488a46bab30cce31410714de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:38:35.477415 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-5e220cc2-90db-44c6-a70d-7d7fd2857922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:38:35.564564 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1130/2286] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:38:35 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:38:35.576684 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318c6b68-d4ed-4e09-bfbe-6490c0e76d7d admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:38:35.579108 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-318c6b68-d4ed-4e09-bfbe-6490c0e76d7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:38:35.656005 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318c6b68-d4ed-4e09-bfbe-6490c0e76d7d admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:38:35.660381 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3386a18-ce96-43e9-ade8-af3dbab207a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd49388facdf446885e55b45ef91f01e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:38:35.661328 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3386a18-ce96-43e9-ade8-af3dbab207a0 service cinder] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:38:35.661657 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3386a18-ce96-43e9-ade8-af3dbab207a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:38:35.661902 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3386a18-ce96-43e9-ade8-af3dbab207a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:38:35.672662 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3386a18-ce96-43e9-ade8-af3dbab207a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:38:35.674145 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3386a18-ce96-43e9-ade8-af3dbab207a0 service cinder] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:38:35.685052 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1157/2287] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:38:35 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:38:43.029888 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27265ebd-f3e7-4c17-8625-9f0cacf47bac None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:38:43.030462 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27265ebd-f3e7-4c17-8625-9f0cacf47bac None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:38:43.030462 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27265ebd-f3e7-4c17-8625-9f0cacf47bac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:38:43.050336 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-27265ebd-f3e7-4c17-8625-9f0cacf47bac None None] MFA Rules not processed for user `768f872096584d4093fc616d547aabb6`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:38:43.051648 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-27265ebd-f3e7-4c17-8625-9f0cacf47bac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:38:43.112247 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1131/2288] 10.222.0.66 () {58 vars in 1009 bytes} [Mon Jul 18 02:38:43 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:38:43.137009 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3386a18-ce96-43e9-ade8-af3dbab207a0 service cinder] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:38:43.142971 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-d3386a18-ce96-43e9-ade8-af3dbab207a0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:38:43.204485 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:38:43.224133 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3386a18-ce96-43e9-ade8-af3dbab207a0 service cinder] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:38:43.229794 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74313df6-5744-4c71-9614-a5cd215712de service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4300622cbd347dd8f5227bb960e98d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:38:43.230574 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74313df6-5744-4c71-9614-a5cd215712de service swift] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:38:43.234462 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74313df6-5744-4c71-9614-a5cd215712de service swift] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:38:43.234462 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74313df6-5744-4c71-9614-a5cd215712de service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:38:43.239267 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74313df6-5744-4c71-9614-a5cd215712de service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:38:43.240886 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74313df6-5744-4c71-9614-a5cd215712de service swift] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:38:43.249663 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1158/2289] 10.222.0.66 () {60 vars in 1384 bytes} [Mon Jul 18 02:38:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 02:38:44.267769 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14179cba-23d4-4e48-a8dc-0dbec280f842 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:38:44.268250 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14179cba-23d4-4e48-a8dc-0dbec280f842 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:38:44.268357 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14179cba-23d4-4e48-a8dc-0dbec280f842 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:38:44.285662 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-14179cba-23d4-4e48-a8dc-0dbec280f842 None None] MFA Rules not processed for user `768f872096584d4093fc616d547aabb6`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:38:44.286836 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-14179cba-23d4-4e48-a8dc-0dbec280f842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:38:44.294225 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1132/2290] 10.222.0.66 () {58 vars in 967 bytes} [Mon Jul 18 02:38:44 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:38:44.306440 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74313df6-5744-4c71-9614-a5cd215712de service swift] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:38:44.317632 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74313df6-5744-4c71-9614-a5cd215712de service swift] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:38:44.322905 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9870d39-d952-417b-b6bb-46c0d7725cfa service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4300622cbd347dd8f5227bb960e98d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:38:44.324203 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9870d39-d952-417b-b6bb-46c0d7725cfa service swift] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:38:44.324489 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9870d39-d952-417b-b6bb-46c0d7725cfa service swift] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:38:44.324768 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9870d39-d952-417b-b6bb-46c0d7725cfa service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:38:44.332961 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9870d39-d952-417b-b6bb-46c0d7725cfa service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:38:44.334706 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9870d39-d952-417b-b6bb-46c0d7725cfa service swift] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:38:44.343277 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1159/2291] 10.222.0.66 () {60 vars in 1384 bytes} [Mon Jul 18 02:38:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 02:38:44.975719 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3e5e66-42f4-4360-9a5d-48d415a54abd None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:38:44.976007 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3e5e66-42f4-4360-9a5d-48d415a54abd None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:38:44.976284 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3e5e66-42f4-4360-9a5d-48d415a54abd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:38:44.999328 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-3f3e5e66-42f4-4360-9a5d-48d415a54abd None None] MFA Rules not processed for user `768f872096584d4093fc616d547aabb6`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:38:45.001078 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-3f3e5e66-42f4-4360-9a5d-48d415a54abd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:38:45.015917 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1133/2292] 10.222.0.66 () {58 vars in 1009 bytes} [Mon Jul 18 02:38:44 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:38:45.029045 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9870d39-d952-417b-b6bb-46c0d7725cfa service swift] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:38:45.046834 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9870d39-d952-417b-b6bb-46c0d7725cfa service swift] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:38:45.055437 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1107348-7fcc-43ab-b8f0-b23232a0a870 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4300622cbd347dd8f5227bb960e98d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:38:45.056209 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1107348-7fcc-43ab-b8f0-b23232a0a870 service swift] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:38:45.056594 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1107348-7fcc-43ab-b8f0-b23232a0a870 service swift] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:38:45.058267 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1107348-7fcc-43ab-b8f0-b23232a0a870 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:38:45.068182 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1107348-7fcc-43ab-b8f0-b23232a0a870 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:38:45.069978 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1107348-7fcc-43ab-b8f0-b23232a0a870 service swift] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:38:45.086463 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1160/2293] 10.222.0.66 () {60 vars in 1384 bytes} [Mon Jul 18 02:38:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 02:38:45.727715 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c961fd-a4f3-4c77-a85a-5e8f7d5124ea None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:38:45.728199 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c961fd-a4f3-4c77-a85a-5e8f7d5124ea None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:38:45.728292 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c961fd-a4f3-4c77-a85a-5e8f7d5124ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:38:45.746133 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-b5c961fd-a4f3-4c77-a85a-5e8f7d5124ea None None] MFA Rules not processed for user `768f872096584d4093fc616d547aabb6`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:38:45.747371 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-b5c961fd-a4f3-4c77-a85a-5e8f7d5124ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:38:45.754550 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1134/2294] 10.222.0.66 () {58 vars in 967 bytes} [Mon Jul 18 02:38:45 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:38:45.766495 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1107348-7fcc-43ab-b8f0-b23232a0a870 service swift] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:38:45.778061 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1107348-7fcc-43ab-b8f0-b23232a0a870 service swift] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:38:45.783087 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc07bda1-8862-4f2a-bbfc-495f2eb93575 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4300622cbd347dd8f5227bb960e98d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:38:45.783723 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc07bda1-8862-4f2a-bbfc-495f2eb93575 service swift] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:38:45.784030 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc07bda1-8862-4f2a-bbfc-495f2eb93575 service swift] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:38:45.784314 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc07bda1-8862-4f2a-bbfc-495f2eb93575 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:38:45.793310 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc07bda1-8862-4f2a-bbfc-495f2eb93575 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:38:45.795177 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc07bda1-8862-4f2a-bbfc-495f2eb93575 service swift] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:38:45.803367 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1161/2295] 10.222.0.66 () {60 vars in 1384 bytes} [Mon Jul 18 02:38:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 02:38:48.640826 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c961fd-a4f3-4c77-a85a-5e8f7d5124ea None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:38:48.642900 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-b5c961fd-a4f3-4c77-a85a-5e8f7d5124ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:38:48.688779 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:38:48.723903 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c961fd-a4f3-4c77-a85a-5e8f7d5124ea None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:38:48.728299 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2428b6fb-8828-484f-b44f-ef3afc2a07f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:38:48.729168 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2428b6fb-8828-484f-b44f-ef3afc2a07f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:38:48.730127 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2428b6fb-8828-484f-b44f-ef3afc2a07f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:38:48.730127 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2428b6fb-8828-484f-b44f-ef3afc2a07f2 admin admin] PATH_INFO: `/v3/users/273664a60a334dc58ede22f9ccc0a128` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:38:48.731618 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2428b6fb-8828-484f-b44f-ef3afc2a07f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=273664a60a334dc58ede22f9ccc0a128)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:38:48.733862 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2428b6fb-8828-484f-b44f-ef3afc2a07f2 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:38:48.962233 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-2428b6fb-8828-484f-b44f-ef3afc2a07f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '273664a60a334dc58ede22f9ccc0a128'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:38:49.077253 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-2428b6fb-8828-484f-b44f-ef3afc2a07f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '273664a60a334dc58ede22f9ccc0a128'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:38:49.077885 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-2428b6fb-8828-484f-b44f-ef3afc2a07f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '273664a60a334dc58ede22f9ccc0a128'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:38:49.090705 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-2428b6fb-8828-484f-b44f-ef3afc2a07f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '273664a60a334dc58ede22f9ccc0a128'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:38:49.091339 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-2428b6fb-8828-484f-b44f-ef3afc2a07f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '273664a60a334dc58ede22f9ccc0a128'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:38:49.103511 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1135/2296] 10.222.0.66 () {58 vars in 1223 bytes} [Mon Jul 18 02:38:48 2022] DELETE /identity/v3/users/273664a60a334dc58ede22f9ccc0a128 => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:38:49.108744 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d99f0e-aa91-4b13-ae21-7edadb98a73b None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:38:49.109231 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d99f0e-aa91-4b13-ae21-7edadb98a73b None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:38:49.109377 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d99f0e-aa91-4b13-ae21-7edadb98a73b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:38:49.131764 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-55d99f0e-aa91-4b13-ae21-7edadb98a73b None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:38:49.131764 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-55d99f0e-aa91-4b13-ae21-7edadb98a73b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:38:49.186156 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1162/2297] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:38:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 02:38:49.195740 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2428b6fb-8828-484f-b44f-ef3afc2a07f2 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:38:49.197159 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-2428b6fb-8828-484f-b44f-ef3afc2a07f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:38:49.253459 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2428b6fb-8828-484f-b44f-ef3afc2a07f2 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:38:49.256994 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506fdbe8-ea82-4250-a258-d64d7e550fc5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eaf19a259c2741cea375a38763e33cdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:38:49.257634 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506fdbe8-ea82-4250-a258-d64d7e550fc5 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:38:49.257936 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506fdbe8-ea82-4250-a258-d64d7e550fc5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:38:49.258242 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506fdbe8-ea82-4250-a258-d64d7e550fc5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:38:49.267546 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506fdbe8-ea82-4250-a258-d64d7e550fc5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:38:49.268989 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506fdbe8-ea82-4250-a258-d64d7e550fc5 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:38:49.278347 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1136/2298] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:38:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:38:49.536081 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d99f0e-aa91-4b13-ae21-7edadb98a73b None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:38:49.537548 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-55d99f0e-aa91-4b13-ae21-7edadb98a73b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:38:49.551447 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d99f0e-aa91-4b13-ae21-7edadb98a73b None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:38:49.556251 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f852d6-fd19-426a-9d38-9b937a2f1ce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:38:49.557260 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f852d6-fd19-426a-9d38-9b937a2f1ce1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:38:49.557736 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f852d6-fd19-426a-9d38-9b937a2f1ce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:38:49.558030 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f852d6-fd19-426a-9d38-9b937a2f1ce1 admin admin] PATH_INFO: `/v3/projects/ba114b0e00424785a219aa31d987d2ee` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:38:49.566108 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f852d6-fd19-426a-9d38-9b937a2f1ce1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba114b0e00424785a219aa31d987d2ee)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:38:49.568111 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f852d6-fd19-426a-9d38-9b937a2f1ce1 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:38:49.745678 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-e0f852d6-fd19-426a-9d38-9b937a2f1ce1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba114b0e00424785a219aa31d987d2ee', 'request_id': 'req-e0f852d6-fd19-426a-9d38-9b937a2f1ce1'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:38:49.764550 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-e0f852d6-fd19-426a-9d38-9b937a2f1ce1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba114b0e00424785a219aa31d987d2ee', 'request_id': 'req-e0f852d6-fd19-426a-9d38-9b937a2f1ce1'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:38:49.890512 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-e0f852d6-fd19-426a-9d38-9b937a2f1ce1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba114b0e00424785a219aa31d987d2ee', 'request_id': 'req-e0f852d6-fd19-426a-9d38-9b937a2f1ce1'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:38:49.903332 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-e0f852d6-fd19-426a-9d38-9b937a2f1ce1 admin admin] The token cache is being invalidate because project ba114b0e00424785a219aa31d987d2ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 02:38:49.903626 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-e0f852d6-fd19-426a-9d38-9b937a2f1ce1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:38:49.904333 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-e0f852d6-fd19-426a-9d38-9b937a2f1ce1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:38:49.906219 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1163/2299] 10.222.0.66 () {58 vars in 1232 bytes} [Mon Jul 18 02:38:49 2022] DELETE /identity/v3/projects/ba114b0e00424785a219aa31d987d2ee => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:38:56.758483 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfb5d4f-601d-4b09-87ef-52319fd1f4e1 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:38:56.759001 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfb5d4f-601d-4b09-87ef-52319fd1f4e1 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:38:56.759126 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfb5d4f-601d-4b09-87ef-52319fd1f4e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:38:56.784737 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-7cfb5d4f-601d-4b09-87ef-52319fd1f4e1 None None] MFA Rules not processed for user `768f872096584d4093fc616d547aabb6`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:38:56.786011 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-7cfb5d4f-601d-4b09-87ef-52319fd1f4e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:38:56.841095 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:38:56.849799 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1137/2300] 10.222.0.66 () {58 vars in 967 bytes} [Mon Jul 18 02:38:56 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:38:56.866315 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f852d6-fd19-426a-9d38-9b937a2f1ce1 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:38:56.866315 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-e0f852d6-fd19-426a-9d38-9b937a2f1ce1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:38:56.884097 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:38:56.958929 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f852d6-fd19-426a-9d38-9b937a2f1ce1 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:38:56.967083 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3985bbbd-8067-484c-b129-19ed6cbf92e2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4300622cbd347dd8f5227bb960e98d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:38:56.967790 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3985bbbd-8067-484c-b129-19ed6cbf92e2 service swift] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:38:56.970340 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3985bbbd-8067-484c-b129-19ed6cbf92e2 service swift] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:38:56.970777 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3985bbbd-8067-484c-b129-19ed6cbf92e2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:38:56.981861 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3985bbbd-8067-484c-b129-19ed6cbf92e2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:38:56.988990 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3985bbbd-8067-484c-b129-19ed6cbf92e2 service swift] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:38:56.998289 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1164/2301] 10.222.0.66 () {60 vars in 1384 bytes} [Mon Jul 18 02:38:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 02:38:57.415190 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfb5d4f-601d-4b09-87ef-52319fd1f4e1 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:38:57.416646 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-7cfb5d4f-601d-4b09-87ef-52319fd1f4e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:38:57.497304 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfb5d4f-601d-4b09-87ef-52319fd1f4e1 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:38:57.502062 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd7c440-6d45-4804-ac3b-7b76d6c614b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:38:57.503059 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd7c440-6d45-4804-ac3b-7b76d6c614b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:38:57.503349 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd7c440-6d45-4804-ac3b-7b76d6c614b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:38:57.503626 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd7c440-6d45-4804-ac3b-7b76d6c614b7 admin admin] PATH_INFO: `/v3/users/2edc56fb488a46bab30cce31410714de` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:38:57.504873 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd7c440-6d45-4804-ac3b-7b76d6c614b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2edc56fb488a46bab30cce31410714de)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:38:57.506840 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd7c440-6d45-4804-ac3b-7b76d6c614b7 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:38:57.721749 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-5dd7c440-6d45-4804-ac3b-7b76d6c614b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2edc56fb488a46bab30cce31410714de'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:38:57.856061 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-5dd7c440-6d45-4804-ac3b-7b76d6c614b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2edc56fb488a46bab30cce31410714de'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:38:57.856666 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-5dd7c440-6d45-4804-ac3b-7b76d6c614b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2edc56fb488a46bab30cce31410714de'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:38:57.870159 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-5dd7c440-6d45-4804-ac3b-7b76d6c614b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2edc56fb488a46bab30cce31410714de'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:38:57.870870 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-5dd7c440-6d45-4804-ac3b-7b76d6c614b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2edc56fb488a46bab30cce31410714de'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:38:57.883262 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1138/2302] 10.222.0.66 () {58 vars in 1223 bytes} [Mon Jul 18 02:38:57 2022] DELETE /identity/v3/users/2edc56fb488a46bab30cce31410714de => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:38:57.888061 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e448c353-4808-4cdc-aed9-5a2c6c551ac2 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:38:57.891049 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e448c353-4808-4cdc-aed9-5a2c6c551ac2 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:38:57.891049 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e448c353-4808-4cdc-aed9-5a2c6c551ac2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:38:57.907334 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-e448c353-4808-4cdc-aed9-5a2c6c551ac2 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:38:57.908641 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-e448c353-4808-4cdc-aed9-5a2c6c551ac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:38:57.970798 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1165/2303] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:38:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:38:57.980790 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd7c440-6d45-4804-ac3b-7b76d6c614b7 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:38:57.982523 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-5dd7c440-6d45-4804-ac3b-7b76d6c614b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:38:58.042223 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd7c440-6d45-4804-ac3b-7b76d6c614b7 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:38:58.046753 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12618a82-b13f-438a-a8c4-ad77d826e478 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eaf19a259c2741cea375a38763e33cdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:38:58.047398 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12618a82-b13f-438a-a8c4-ad77d826e478 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:38:58.047683 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12618a82-b13f-438a-a8c4-ad77d826e478 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:38:58.047963 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12618a82-b13f-438a-a8c4-ad77d826e478 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:38:58.056975 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12618a82-b13f-438a-a8c4-ad77d826e478 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:38:58.059167 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12618a82-b13f-438a-a8c4-ad77d826e478 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:38:58.074811 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1139/2304] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:38:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:38:58.620146 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e448c353-4808-4cdc-aed9-5a2c6c551ac2 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:38:58.622218 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-e448c353-4808-4cdc-aed9-5a2c6c551ac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:38:58.634535 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e448c353-4808-4cdc-aed9-5a2c6c551ac2 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:38:58.640531 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cafb288-45cc-4e26-a8a6-3b5c46c722fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:38:58.641409 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cafb288-45cc-4e26-a8a6-3b5c46c722fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:38:58.641694 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cafb288-45cc-4e26-a8a6-3b5c46c722fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:38:58.642030 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cafb288-45cc-4e26-a8a6-3b5c46c722fe admin admin] PATH_INFO: `/v3/projects/4d6bd46b2dcb4bea8c4215742c4e70d3` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:38:58.643241 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cafb288-45cc-4e26-a8a6-3b5c46c722fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d6bd46b2dcb4bea8c4215742c4e70d3)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:38:58.645159 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cafb288-45cc-4e26-a8a6-3b5c46c722fe admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:38:58.799688 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-6cafb288-45cc-4e26-a8a6-3b5c46c722fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d6bd46b2dcb4bea8c4215742c4e70d3', 'request_id': 'req-6cafb288-45cc-4e26-a8a6-3b5c46c722fe'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:38:58.805021 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-6cafb288-45cc-4e26-a8a6-3b5c46c722fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d6bd46b2dcb4bea8c4215742c4e70d3', 'request_id': 'req-6cafb288-45cc-4e26-a8a6-3b5c46c722fe'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:38:58.901525 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-6cafb288-45cc-4e26-a8a6-3b5c46c722fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d6bd46b2dcb4bea8c4215742c4e70d3', 'request_id': 'req-6cafb288-45cc-4e26-a8a6-3b5c46c722fe'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:38:58.913252 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-6cafb288-45cc-4e26-a8a6-3b5c46c722fe admin admin] The token cache is being invalidate because project 4d6bd46b2dcb4bea8c4215742c4e70d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 02:38:58.916565 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-6cafb288-45cc-4e26-a8a6-3b5c46c722fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:38:58.916565 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-6cafb288-45cc-4e26-a8a6-3b5c46c722fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:38:58.916565 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1166/2305] 10.222.0.66 () {58 vars in 1232 bytes} [Mon Jul 18 02:38:58 2022] DELETE /identity/v3/projects/4d6bd46b2dcb4bea8c4215742c4e70d3 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:39:06.927065 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12618a82-b13f-438a-a8c4-ad77d826e478 service neutron] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:39:06.936259 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12618a82-b13f-438a-a8c4-ad77d826e478 service neutron] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:39:06.952185 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:39:06.983604 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452924ef-e30b-4660-9a62-580af9560a5a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eaf19a259c2741cea375a38763e33cdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:39:06.984441 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452924ef-e30b-4660-9a62-580af9560a5a service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:39:06.984721 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452924ef-e30b-4660-9a62-580af9560a5a service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:39:06.984989 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452924ef-e30b-4660-9a62-580af9560a5a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:39:06.986773 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-452924ef-e30b-4660-9a62-580af9560a5a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:39:07.049776 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452924ef-e30b-4660-9a62-580af9560a5a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:39:07.051196 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452924ef-e30b-4660-9a62-580af9560a5a service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:39:07.062289 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1140/2306] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:39:06 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:39:08.910332 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=80,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:39:11.969395 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef75cb00-02da-41a4-b7cf-0191e2dbb421 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:39:11.970204 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef75cb00-02da-41a4-b7cf-0191e2dbb421 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:39:11.970204 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef75cb00-02da-41a4-b7cf-0191e2dbb421 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:39:11.988061 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-ef75cb00-02da-41a4-b7cf-0191e2dbb421 None None] MFA Rules not processed for user `94c16391d39f4498a55c89bf2838788f`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:39:11.990412 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-ef75cb00-02da-41a4-b7cf-0191e2dbb421 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:39:12.043940 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1167/2307] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:39:11 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:39:17.102620 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=41 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:39:22.037192 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=25 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:39:51.001688 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db543504-caf2-470a-bfff-00e3e73a8e3d None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:39:51.002813 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db543504-caf2-470a-bfff-00e3e73a8e3d None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:39:51.002813 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db543504-caf2-470a-bfff-00e3e73a8e3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:39:51.024454 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-db543504-caf2-470a-bfff-00e3e73a8e3d None None] MFA Rules not processed for user `7ddb197852db4b20a84aa6c0862468ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:39:51.025673 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-db543504-caf2-470a-bfff-00e3e73a8e3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:39:51.080293 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aabe16-e43c-4d61-a476-f5a360f19e6c None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:39:51.080860 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aabe16-e43c-4d61-a476-f5a360f19e6c None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:39:51.080965 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aabe16-e43c-4d61-a476-f5a360f19e6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:39:51.085859 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1141/2308] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:39:50 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:39:51.100588 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-31aabe16-e43c-4d61-a476-f5a360f19e6c None None] MFA Rules not processed for user `d281c92e20d04004a679a8ec65f9e6d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:39:51.101813 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-31aabe16-e43c-4d61-a476-f5a360f19e6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:39:51.162255 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1168/2309] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:39:51 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 02:40:01.028769 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db543504-caf2-470a-bfff-00e3e73a8e3d None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:40:01.030293 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-db543504-caf2-470a-bfff-00e3e73a8e3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:40:01.042755 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db543504-caf2-470a-bfff-00e3e73a8e3d None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:40:01.046537 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfae0f70-a9ae-45fc-85a2-dd0189ffc220 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eaf19a259c2741cea375a38763e33cdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:40:01.047263 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfae0f70-a9ae-45fc-85a2-dd0189ffc220 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:40:01.047551 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfae0f70-a9ae-45fc-85a2-dd0189ffc220 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:40:01.047825 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfae0f70-a9ae-45fc-85a2-dd0189ffc220 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:40:01.049588 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-cfae0f70-a9ae-45fc-85a2-dd0189ffc220 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:40:01.121054 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfae0f70-a9ae-45fc-85a2-dd0189ffc220 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:40:01.122529 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfae0f70-a9ae-45fc-85a2-dd0189ffc220 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:40:01.133328 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1142/2310] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:40:01 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:40:01.154672 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=25 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:40:02.309641 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31aabe16-e43c-4d61-a476-f5a360f19e6c None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:40:02.311148 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-31aabe16-e43c-4d61-a476-f5a360f19e6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:40:02.323468 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31aabe16-e43c-4d61-a476-f5a360f19e6c None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:40:02.327030 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c29de51-ada9-483e-b6b3-b4d355de5a6b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eaf19a259c2741cea375a38763e33cdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:40:02.327670 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c29de51-ada9-483e-b6b3-b4d355de5a6b service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:40:02.327956 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c29de51-ada9-483e-b6b3-b4d355de5a6b service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:40:02.328238 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c29de51-ada9-483e-b6b3-b4d355de5a6b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:40:02.329956 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-9c29de51-ada9-483e-b6b3-b4d355de5a6b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:40:02.343589 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c29de51-ada9-483e-b6b3-b4d355de5a6b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:40:02.345018 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c29de51-ada9-483e-b6b3-b4d355de5a6b service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:40:02.355984 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1169/2311] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:40:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:40:11.128472 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=59 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:40:12.169795 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfae0f70-a9ae-45fc-85a2-dd0189ffc220 service neutron] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:40:12.171240 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-cfae0f70-a9ae-45fc-85a2-dd0189ffc220 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:40:12.228613 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfae0f70-a9ae-45fc-85a2-dd0189ffc220 service neutron] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:40:12.232520 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfc4b60-68c8-4afd-bed7-2e456f6c4c38 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd49388facdf446885e55b45ef91f01e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:40:12.233213 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfc4b60-68c8-4afd-bed7-2e456f6c4c38 service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:40:12.233500 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfc4b60-68c8-4afd-bed7-2e456f6c4c38 service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:40:12.233780 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfc4b60-68c8-4afd-bed7-2e456f6c4c38 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:40:12.235735 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-7dfc4b60-68c8-4afd-bed7-2e456f6c4c38 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:40:12.298876 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfc4b60-68c8-4afd-bed7-2e456f6c4c38 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:40:12.300384 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfc4b60-68c8-4afd-bed7-2e456f6c4c38 service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:40:12.311045 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1143/2312] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:40:12 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:40:12.351155 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=6 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:40:17.486406 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c29de51-ada9-483e-b6b3-b4d355de5a6b service neutron] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:40:17.487904 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-9c29de51-ada9-483e-b6b3-b4d355de5a6b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:40:17.499371 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c29de51-ada9-483e-b6b3-b4d355de5a6b service neutron] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:40:17.502941 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65371e6-4dc6-4288-a96d-6dd9fec7d0af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd49388facdf446885e55b45ef91f01e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:40:17.503569 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65371e6-4dc6-4288-a96d-6dd9fec7d0af service cinder] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:40:17.503851 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65371e6-4dc6-4288-a96d-6dd9fec7d0af service cinder] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:40:17.504123 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65371e6-4dc6-4288-a96d-6dd9fec7d0af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:40:17.506137 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-f65371e6-4dc6-4288-a96d-6dd9fec7d0af service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:40:17.517141 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65371e6-4dc6-4288-a96d-6dd9fec7d0af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:40:17.518636 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65371e6-4dc6-4288-a96d-6dd9fec7d0af service cinder] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:40:17.528965 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1170/2313] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:40:17 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:40:21.046332 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6971919-4e7a-4960-9fc6-7a105018b8e7 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:40:21.046835 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6971919-4e7a-4960-9fc6-7a105018b8e7 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:40:21.046946 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6971919-4e7a-4960-9fc6-7a105018b8e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:40:21.064408 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-d6971919-4e7a-4960-9fc6-7a105018b8e7 None None] MFA Rules not processed for user `879f7a4cf6664e76bddffc804658b250`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:40:21.065704 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-d6971919-4e7a-4960-9fc6-7a105018b8e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:40:21.073161 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1144/2314] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:40:21 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:40:21.377617 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65371e6-4dc6-4288-a96d-6dd9fec7d0af service cinder] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:40:21.387123 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65371e6-4dc6-4288-a96d-6dd9fec7d0af service cinder] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:40:21.390753 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e29227d-336d-4e80-b616-365fa6e33c79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eaf19a259c2741cea375a38763e33cdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:40:21.391392 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e29227d-336d-4e80-b616-365fa6e33c79 service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:40:21.391679 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e29227d-336d-4e80-b616-365fa6e33c79 service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:40:21.391964 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e29227d-336d-4e80-b616-365fa6e33c79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:40:21.393718 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-7e29227d-336d-4e80-b616-365fa6e33c79 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:40:21.403903 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e29227d-336d-4e80-b616-365fa6e33c79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:40:21.405325 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e29227d-336d-4e80-b616-365fa6e33c79 service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:40:21.414118 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1171/2315] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:40:21 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:40:21.851884 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6971919-4e7a-4960-9fc6-7a105018b8e7 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:40:21.853320 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-d6971919-4e7a-4960-9fc6-7a105018b8e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:40:21.924123 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6971919-4e7a-4960-9fc6-7a105018b8e7 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:40:21.929942 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b769792c-8923-45b3-8a9b-7d3733afbbd6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88936b31709b429f8b6777c673ed7b8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:40:21.930621 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b769792c-8923-45b3-8a9b-7d3733afbbd6 service placement] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:40:21.930906 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b769792c-8923-45b3-8a9b-7d3733afbbd6 service placement] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:40:21.931183 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b769792c-8923-45b3-8a9b-7d3733afbbd6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:40:21.932953 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-b769792c-8923-45b3-8a9b-7d3733afbbd6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:40:21.944770 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b769792c-8923-45b3-8a9b-7d3733afbbd6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:40:21.946418 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b769792c-8923-45b3-8a9b-7d3733afbbd6 service placement] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:40:21.956168 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1145/2316] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:40:21 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 02:40:26.210716 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e29227d-336d-4e80-b616-365fa6e33c79 service neutron] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:40:26.219476 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e29227d-336d-4e80-b616-365fa6e33c79 service neutron] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:40:26.223056 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb767465-d63c-42e8-a227-b7b2b85d2889 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eaf19a259c2741cea375a38763e33cdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:40:26.223725 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb767465-d63c-42e8-a227-b7b2b85d2889 service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:40:26.224009 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb767465-d63c-42e8-a227-b7b2b85d2889 service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:40:26.224287 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb767465-d63c-42e8-a227-b7b2b85d2889 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:40:26.226057 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-eb767465-d63c-42e8-a227-b7b2b85d2889 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:40:26.235990 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb767465-d63c-42e8-a227-b7b2b85d2889 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:40:26.237428 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb767465-d63c-42e8-a227-b7b2b85d2889 service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:40:26.245766 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1172/2317] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:40:26 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:40:31.952047 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=83 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:40:36.242322 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=18 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:40:36.496324 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4519ea8a-78b2-4d43-b5fc-b926a6c83009 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:40:36.496831 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4519ea8a-78b2-4d43-b5fc-b926a6c83009 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:40:36.496931 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4519ea8a-78b2-4d43-b5fc-b926a6c83009 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:40:36.515296 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-4519ea8a-78b2-4d43-b5fc-b926a6c83009 None None] MFA Rules not processed for user `94c16391d39f4498a55c89bf2838788f`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:40:36.516578 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-4519ea8a-78b2-4d43-b5fc-b926a6c83009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:40:36.524477 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1146/2318] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:40:36 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:40:36.864205 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb767465-d63c-42e8-a227-b7b2b85d2889 service neutron] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:40:36.873725 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb767465-d63c-42e8-a227-b7b2b85d2889 service neutron] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:40:36.877391 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f7d17e-e486-4da8-89b3-bc46364d1863 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eaf19a259c2741cea375a38763e33cdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:40:36.878030 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f7d17e-e486-4da8-89b3-bc46364d1863 service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:40:36.878345 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f7d17e-e486-4da8-89b3-bc46364d1863 service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:40:36.878615 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f7d17e-e486-4da8-89b3-bc46364d1863 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:40:36.880485 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-78f7d17e-e486-4da8-89b3-bc46364d1863 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:40:36.890328 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f7d17e-e486-4da8-89b3-bc46364d1863 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:40:36.891753 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f7d17e-e486-4da8-89b3-bc46364d1863 service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:40:36.900234 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1173/2319] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:40:36 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:40:42.069191 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4519ea8a-78b2-4d43-b5fc-b926a6c83009 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:40:42.071373 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-4519ea8a-78b2-4d43-b5fc-b926a6c83009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:40:42.100860 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4519ea8a-78b2-4d43-b5fc-b926a6c83009 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:40:42.125313 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f08dfd-0e14-4c03-9130-f94a75297ba3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80b138831bc54c7b986d50092d62d4af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:40:42.125996 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f08dfd-0e14-4c03-9130-f94a75297ba3 service nova] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:40:42.126319 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f08dfd-0e14-4c03-9130-f94a75297ba3 service nova] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:40:42.126591 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f08dfd-0e14-4c03-9130-f94a75297ba3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:40:42.130020 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-95f08dfd-0e14-4c03-9130-f94a75297ba3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:40:42.153380 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f08dfd-0e14-4c03-9130-f94a75297ba3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:40:42.156731 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f08dfd-0e14-4c03-9130-f94a75297ba3 service nova] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:40:42.173673 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1147/2320] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:40:42 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:40:46.896608 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=6 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:40:50.001489 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f7d17e-e486-4da8-89b3-bc46364d1863 service neutron] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:40:50.003925 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-78f7d17e-e486-4da8-89b3-bc46364d1863 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:40:50.021559 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f7d17e-e486-4da8-89b3-bc46364d1863 service neutron] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:40:50.027274 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b1a110-e5a5-4056-9d98-ed358ecd9d11 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80b138831bc54c7b986d50092d62d4af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:40:50.027986 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b1a110-e5a5-4056-9d98-ed358ecd9d11 service nova] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:40:50.028261 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b1a110-e5a5-4056-9d98-ed358ecd9d11 service nova] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:40:50.028534 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b1a110-e5a5-4056-9d98-ed358ecd9d11 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:40:50.030402 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-f1b1a110-e5a5-4056-9d98-ed358ecd9d11 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:40:50.043879 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b1a110-e5a5-4056-9d98-ed358ecd9d11 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:40:50.045508 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b1a110-e5a5-4056-9d98-ed358ecd9d11 service nova] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:40:50.057650 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1174/2321] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:40:50 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:40:50.753051 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f08dfd-0e14-4c03-9130-f94a75297ba3 service nova] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:40:50.766004 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f08dfd-0e14-4c03-9130-f94a75297ba3 service nova] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:40:50.772341 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892488ce-159c-4c42-bf72-f72583c74bae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80b138831bc54c7b986d50092d62d4af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:40:50.772997 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892488ce-159c-4c42-bf72-f72583c74bae service nova] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:40:50.773285 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892488ce-159c-4c42-bf72-f72583c74bae service nova] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:40:50.773567 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892488ce-159c-4c42-bf72-f72583c74bae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:40:50.775415 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-892488ce-159c-4c42-bf72-f72583c74bae service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:40:50.786220 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892488ce-159c-4c42-bf72-f72583c74bae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:40:50.787805 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892488ce-159c-4c42-bf72-f72583c74bae service nova] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:40:50.796911 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1148/2322] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:40:50 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:40:55.222300 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b1a110-e5a5-4056-9d98-ed358ecd9d11 service nova] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:40:55.234025 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b1a110-e5a5-4056-9d98-ed358ecd9d11 service nova] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:40:55.240113 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfef780-e60c-4462-8cb9-5a348001c6d6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80b138831bc54c7b986d50092d62d4af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:40:55.240849 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfef780-e60c-4462-8cb9-5a348001c6d6 service nova] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:40:55.241142 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfef780-e60c-4462-8cb9-5a348001c6d6 service nova] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:40:55.241425 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfef780-e60c-4462-8cb9-5a348001c6d6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:40:55.244994 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-9dfef780-e60c-4462-8cb9-5a348001c6d6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:40:55.260798 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfef780-e60c-4462-8cb9-5a348001c6d6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:40:55.262805 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfef780-e60c-4462-8cb9-5a348001c6d6 service nova] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:40:55.275027 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1175/2323] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:40:55 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:41:00.792608 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=15 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:41:04.339100 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892488ce-159c-4c42-bf72-f72583c74bae service nova] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:41:04.356097 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892488ce-159c-4c42-bf72-f72583c74bae service nova] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:41:04.363387 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25907cd5-f062-4677-a88c-9b318f926335 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88936b31709b429f8b6777c673ed7b8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:41:04.364173 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25907cd5-f062-4677-a88c-9b318f926335 service placement] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:41:04.364562 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25907cd5-f062-4677-a88c-9b318f926335 service placement] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:41:04.365702 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25907cd5-f062-4677-a88c-9b318f926335 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:41:04.367124 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-25907cd5-f062-4677-a88c-9b318f926335 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:41:04.380181 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25907cd5-f062-4677-a88c-9b318f926335 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:41:04.381860 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25907cd5-f062-4677-a88c-9b318f926335 service placement] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:41:04.392444 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1149/2324] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:41:04 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:41:04.513489 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfef780-e60c-4462-8cb9-5a348001c6d6 service nova] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:41:04.515142 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-9dfef780-e60c-4462-8cb9-5a348001c6d6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:41:04.580415 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfef780-e60c-4462-8cb9-5a348001c6d6 service nova] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:41:04.585838 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c333e535-1a37-4936-8af2-5da58b962b3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:41:04.586749 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c333e535-1a37-4936-8af2-5da58b962b3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:41:04.587036 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c333e535-1a37-4936-8af2-5da58b962b3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:41:04.587313 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c333e535-1a37-4936-8af2-5da58b962b3f admin admin] PATH_INFO: `/v3/users/d281c92e20d04004a679a8ec65f9e6d5` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:41:04.588511 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c333e535-1a37-4936-8af2-5da58b962b3f admin admin] RBAC: Authorizing `identity:delete_user(user_id=d281c92e20d04004a679a8ec65f9e6d5)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:41:04.590616 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c333e535-1a37-4936-8af2-5da58b962b3f admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:41:04.787898 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-c333e535-1a37-4936-8af2-5da58b962b3f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd281c92e20d04004a679a8ec65f9e6d5'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:41:04.865947 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-c333e535-1a37-4936-8af2-5da58b962b3f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd281c92e20d04004a679a8ec65f9e6d5'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:41:04.873046 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-c333e535-1a37-4936-8af2-5da58b962b3f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd281c92e20d04004a679a8ec65f9e6d5'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:41:04.873683 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-c333e535-1a37-4936-8af2-5da58b962b3f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd281c92e20d04004a679a8ec65f9e6d5'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:41:04.886537 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-c333e535-1a37-4936-8af2-5da58b962b3f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd281c92e20d04004a679a8ec65f9e6d5'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:41:04.892419 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1176/2325] 10.222.0.66 () {58 vars in 1223 bytes} [Mon Jul 18 02:41:04 2022] DELETE /identity/v3/users/d281c92e20d04004a679a8ec65f9e6d5 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:41:05.151463 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25907cd5-f062-4677-a88c-9b318f926335 service placement] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:41:05.169058 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25907cd5-f062-4677-a88c-9b318f926335 service placement] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:41:05.223537 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b3b84f-680b-4fd8-9641-6f3716db87b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:41:05.224396 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b3b84f-680b-4fd8-9641-6f3716db87b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:41:05.224675 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b3b84f-680b-4fd8-9641-6f3716db87b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:41:05.224949 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b3b84f-680b-4fd8-9641-6f3716db87b6 admin admin] PATH_INFO: `/v3/projects/9ec4c1c1687944b98c1c5cefe05df095` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:41:05.226190 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b3b84f-680b-4fd8-9641-6f3716db87b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ec4c1c1687944b98c1c5cefe05df095)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:41:05.228084 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b3b84f-680b-4fd8-9641-6f3716db87b6 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:41:05.379344 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-05b3b84f-680b-4fd8-9641-6f3716db87b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ec4c1c1687944b98c1c5cefe05df095', 'request_id': 'req-05b3b84f-680b-4fd8-9641-6f3716db87b6'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:41:05.385794 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-05b3b84f-680b-4fd8-9641-6f3716db87b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ec4c1c1687944b98c1c5cefe05df095', 'request_id': 'req-05b3b84f-680b-4fd8-9641-6f3716db87b6'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:41:05.392686 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-05b3b84f-680b-4fd8-9641-6f3716db87b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ec4c1c1687944b98c1c5cefe05df095', 'request_id': 'req-05b3b84f-680b-4fd8-9641-6f3716db87b6'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:41:05.481269 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-05b3b84f-680b-4fd8-9641-6f3716db87b6 admin admin] The token cache is being invalidate because project 9ec4c1c1687944b98c1c5cefe05df095 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 02:41:05.481579 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-05b3b84f-680b-4fd8-9641-6f3716db87b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:41:05.482254 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-05b3b84f-680b-4fd8-9641-6f3716db87b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:41:05.483803 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1150/2326] 10.222.0.66 () {58 vars in 1232 bytes} [Mon Jul 18 02:41:05 2022] DELETE /identity/v3/projects/9ec4c1c1687944b98c1c5cefe05df095 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 02:41:05.488852 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46f5622-d86f-4ec6-95ff-b3a4c24bfd4c None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:41:05.489145 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46f5622-d86f-4ec6-95ff-b3a4c24bfd4c None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:41:05.489420 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46f5622-d86f-4ec6-95ff-b3a4c24bfd4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:41:05.507950 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-c46f5622-d86f-4ec6-95ff-b3a4c24bfd4c None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:41:05.509928 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-c46f5622-d86f-4ec6-95ff-b3a4c24bfd4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:41:05.562002 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:41:05.575342 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1177/2327] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:41:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:41:05.583683 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a62423-ba26-40cf-a18f-db943eee895a None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:41:05.584027 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a62423-ba26-40cf-a18f-db943eee895a None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:41:05.584315 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a62423-ba26-40cf-a18f-db943eee895a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:41:05.603244 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-87a62423-ba26-40cf-a18f-db943eee895a None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:41:05.612099 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-87a62423-ba26-40cf-a18f-db943eee895a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:41:05.613365 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1151/2328] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:41:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:41:05.618223 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46f5622-d86f-4ec6-95ff-b3a4c24bfd4c None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:41:05.627930 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46f5622-d86f-4ec6-95ff-b3a4c24bfd4c None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:41:05.633811 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea4b03a-c6f3-4c50-af73-7f58937e4c40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:41:05.634547 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea4b03a-c6f3-4c50-af73-7f58937e4c40 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:41:05.634833 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea4b03a-c6f3-4c50-af73-7f58937e4c40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:41:05.635301 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea4b03a-c6f3-4c50-af73-7f58937e4c40 admin admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:41:05.636068 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea4b03a-c6f3-4c50-af73-7f58937e4c40 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:41:05.637005 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea4b03a-c6f3-4c50-af73-7f58937e4c40 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:41:05.638688 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea4b03a-c6f3-4c50-af73-7f58937e4c40 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:41:05.650988 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1178/2329] 10.222.0.66 () {58 vars in 1152 bytes} [Mon Jul 18 02:41:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:41:05.655475 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a62423-ba26-40cf-a18f-db943eee895a None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:41:05.666222 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a62423-ba26-40cf-a18f-db943eee895a None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:41:05.671398 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51c3fda-8b15-4d66-8c77-684c934f2dc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:41:05.675263 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51c3fda-8b15-4d66-8c77-684c934f2dc5 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:41:05.675263 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51c3fda-8b15-4d66-8c77-684c934f2dc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:41:05.675263 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51c3fda-8b15-4d66-8c77-684c934f2dc5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:41:05.675263 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51c3fda-8b15-4d66-8c77-684c934f2dc5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:41:05.675577 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51c3fda-8b15-4d66-8c77-684c934f2dc5 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:41:05.769260 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1152/2330] 10.222.0.66 () {60 vars in 1152 bytes} [Mon Jul 18 02:41:05 2022] POST /identity/v3/projects => generated 371 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 02:41:05.774018 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea4b03a-c6f3-4c50-af73-7f58937e4c40 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:41:05.784064 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea4b03a-c6f3-4c50-af73-7f58937e4c40 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:41:05.839733 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d0d089-e2fd-4166-9f29-a048050b6436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:41:05.840404 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d0d089-e2fd-4166-9f29-a048050b6436 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:41:05.840691 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d0d089-e2fd-4166-9f29-a048050b6436 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:41:05.840967 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d0d089-e2fd-4166-9f29-a048050b6436 admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:41:05.841666 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d0d089-e2fd-4166-9f29-a048050b6436 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:41:05.843655 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d0d089-e2fd-4166-9f29-a048050b6436 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:41:05.968140 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1179/2331] 10.222.0.66 () {60 vars in 1143 bytes} [Mon Jul 18 02:41:05 2022] POST /identity/v3/users => generated 286 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 02:41:05.969351 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51c3fda-8b15-4d66-8c77-684c934f2dc5 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:41:05.980472 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51c3fda-8b15-4d66-8c77-684c934f2dc5 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:41:05.985068 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caafa01e-9bc6-4016-be8c-98263798cc66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:41:05.985726 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caafa01e-9bc6-4016-be8c-98263798cc66 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:41:05.986009 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caafa01e-9bc6-4016-be8c-98263798cc66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:41:05.986322 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caafa01e-9bc6-4016-be8c-98263798cc66 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:41:05.987041 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caafa01e-9bc6-4016-be8c-98263798cc66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:41:05.988639 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caafa01e-9bc6-4016-be8c-98263798cc66 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:41:06.002592 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1153/2332] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:41:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:41:06.003740 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d0d089-e2fd-4166-9f29-a048050b6436 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:41:06.014005 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d0d089-e2fd-4166-9f29-a048050b6436 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:41:06.019075 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b76bca-8e7c-42f7-a60f-973292071934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:41:06.019737 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b76bca-8e7c-42f7-a60f-973292071934 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:41:06.020027 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b76bca-8e7c-42f7-a60f-973292071934 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:41:06.020309 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b76bca-8e7c-42f7-a60f-973292071934 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:41:06.021032 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b76bca-8e7c-42f7-a60f-973292071934 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:41:06.023256 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b76bca-8e7c-42f7-a60f-973292071934 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:41:06.033448 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1180/2333] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:41:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:41:06.038313 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caafa01e-9bc6-4016-be8c-98263798cc66 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:41:06.048201 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caafa01e-9bc6-4016-be8c-98263798cc66 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:41:06.052799 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44f2d35-80f2-4d0b-9e72-3954a783080c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:41:06.053690 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44f2d35-80f2-4d0b-9e72-3954a783080c admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:41:06.053974 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44f2d35-80f2-4d0b-9e72-3954a783080c admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:41:06.054296 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44f2d35-80f2-4d0b-9e72-3954a783080c admin admin] PATH_INFO: `/v3/projects/8a7d4a7c8a2f4e819675f0800fa3bf40/users/57c838448349443aa08d6dc0c661584d/roles/65feabf3b27944a8b01a86c8ac608457` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:41:06.069914 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44f2d35-80f2-4d0b-9e72-3954a783080c admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a7d4a7c8a2f4e819675f0800fa3bf40, user_id=57c838448349443aa08d6dc0c661584d, role_id=65feabf3b27944a8b01a86c8ac608457)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:41:06.072636 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44f2d35-80f2-4d0b-9e72-3954a783080c admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:41:06.148282 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1154/2334] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:41:06 2022] PUT /identity/v3/projects/8a7d4a7c8a2f4e819675f0800fa3bf40/users/57c838448349443aa08d6dc0c661584d/roles/65feabf3b27944a8b01a86c8ac608457 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:41:06.158597 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27362cd8-0174-4a60-8c5b-2aa815de7e37 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:41:06.158893 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27362cd8-0174-4a60-8c5b-2aa815de7e37 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:41:06.159197 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27362cd8-0174-4a60-8c5b-2aa815de7e37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:41:06.186240 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-27362cd8-0174-4a60-8c5b-2aa815de7e37 None None] MFA Rules not processed for user `57c838448349443aa08d6dc0c661584d`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:41:06.187457 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-27362cd8-0174-4a60-8c5b-2aa815de7e37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:41:06.253053 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1181/2335] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:41:06 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:41:06.258384 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44f2d35-80f2-4d0b-9e72-3954a783080c admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:41:06.273420 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:41:06.324054 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44f2d35-80f2-4d0b-9e72-3954a783080c admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:41:06.328717 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3ed69d-50f4-49f6-90c5-3f3309bc6c12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:41:06.329369 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3ed69d-50f4-49f6-90c5-3f3309bc6c12 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:41:06.329650 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3ed69d-50f4-49f6-90c5-3f3309bc6c12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:41:06.329936 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3ed69d-50f4-49f6-90c5-3f3309bc6c12 admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:41:06.330651 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3ed69d-50f4-49f6-90c5-3f3309bc6c12 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:41:06.332594 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3ed69d-50f4-49f6-90c5-3f3309bc6c12 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:41:06.454323 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1155/2336] 10.222.0.66 () {60 vars in 1152 bytes} [Mon Jul 18 02:41:06 2022] POST /identity/v3/projects => generated 373 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:41:06.456220 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27362cd8-0174-4a60-8c5b-2aa815de7e37 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:41:06.468556 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27362cd8-0174-4a60-8c5b-2aa815de7e37 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:41:06.516069 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:41:06.528623 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebdb33c-bf21-44dc-80f9-7a267c87279a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:41:06.529296 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebdb33c-bf21-44dc-80f9-7a267c87279a admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:41:06.529578 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebdb33c-bf21-44dc-80f9-7a267c87279a admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:41:06.530256 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebdb33c-bf21-44dc-80f9-7a267c87279a admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:41:06.530571 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebdb33c-bf21-44dc-80f9-7a267c87279a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:41:06.532498 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebdb33c-bf21-44dc-80f9-7a267c87279a admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:41:06.630468 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1182/2337] 10.222.0.66 () {60 vars in 1143 bytes} [Mon Jul 18 02:41:06 2022] POST /identity/v3/users => generated 293 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:41:06.634658 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3ed69d-50f4-49f6-90c5-3f3309bc6c12 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:41:06.645069 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3ed69d-50f4-49f6-90c5-3f3309bc6c12 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:41:06.650277 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db977b9b-ce2e-4679-83b1-7bb7766bef7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:41:06.650963 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db977b9b-ce2e-4679-83b1-7bb7766bef7e admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:41:06.651247 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db977b9b-ce2e-4679-83b1-7bb7766bef7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:41:06.651523 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db977b9b-ce2e-4679-83b1-7bb7766bef7e admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:41:06.652245 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db977b9b-ce2e-4679-83b1-7bb7766bef7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:41:06.653870 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db977b9b-ce2e-4679-83b1-7bb7766bef7e admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:41:06.666027 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1156/2338] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:41:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:41:06.669961 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebdb33c-bf21-44dc-80f9-7a267c87279a admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:41:06.679912 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebdb33c-bf21-44dc-80f9-7a267c87279a admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:41:06.684742 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1143b3b4-3b24-4859-9b32-ca43e293b3ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:41:06.685610 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1143b3b4-3b24-4859-9b32-ca43e293b3ee admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:41:06.685896 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1143b3b4-3b24-4859-9b32-ca43e293b3ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:41:06.686203 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1143b3b4-3b24-4859-9b32-ca43e293b3ee admin admin] PATH_INFO: `/v3/projects/67baeb6265c54e25ae0d2aa46980f34d/users/fd3da03710e94cc7bd09a7ee7843b64d/roles/d4aa3324f0c84887b41d0dc4acf7f30e` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:41:06.701748 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1143b3b4-3b24-4859-9b32-ca43e293b3ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=67baeb6265c54e25ae0d2aa46980f34d, user_id=fd3da03710e94cc7bd09a7ee7843b64d, role_id=d4aa3324f0c84887b41d0dc4acf7f30e)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:41:06.704446 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1143b3b4-3b24-4859-9b32-ca43e293b3ee admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:41:06.779676 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1183/2339] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:41:06 2022] PUT /identity/v3/projects/67baeb6265c54e25ae0d2aa46980f34d/users/fd3da03710e94cc7bd09a7ee7843b64d/roles/d4aa3324f0c84887b41d0dc4acf7f30e => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:41:06.783612 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db977b9b-ce2e-4679-83b1-7bb7766bef7e admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:41:06.794189 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db977b9b-ce2e-4679-83b1-7bb7766bef7e admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:41:06.852005 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b928458-8f11-46f6-87f0-49d9349e1819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:41:06.852685 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b928458-8f11-46f6-87f0-49d9349e1819 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:41:06.852971 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b928458-8f11-46f6-87f0-49d9349e1819 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:41:06.853243 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b928458-8f11-46f6-87f0-49d9349e1819 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:41:06.853960 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b928458-8f11-46f6-87f0-49d9349e1819 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:41:06.855605 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b928458-8f11-46f6-87f0-49d9349e1819 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:41:06.865788 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1157/2340] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:41:06 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:41:06.869796 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1143b3b4-3b24-4859-9b32-ca43e293b3ee admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:41:06.879873 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1143b3b4-3b24-4859-9b32-ca43e293b3ee admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:41:06.885090 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ce5b9e-9555-4319-b9f0-b70a8eb6b4ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:41:06.886102 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ce5b9e-9555-4319-b9f0-b70a8eb6b4ef admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:41:06.886395 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ce5b9e-9555-4319-b9f0-b70a8eb6b4ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:41:06.886671 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ce5b9e-9555-4319-b9f0-b70a8eb6b4ef admin admin] PATH_INFO: `/v3/projects/67baeb6265c54e25ae0d2aa46980f34d/users/fd3da03710e94cc7bd09a7ee7843b64d/roles/d4aa3324f0c84887b41d0dc4acf7f30e` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:41:06.888651 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ce5b9e-9555-4319-b9f0-b70a8eb6b4ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=67baeb6265c54e25ae0d2aa46980f34d, user_id=fd3da03710e94cc7bd09a7ee7843b64d, role_id=d4aa3324f0c84887b41d0dc4acf7f30e)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:41:06.891339 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ce5b9e-9555-4319-b9f0-b70a8eb6b4ef admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:41:06.902100 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1184/2341] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:41:06 2022] PUT /identity/v3/projects/67baeb6265c54e25ae0d2aa46980f34d/users/fd3da03710e94cc7bd09a7ee7843b64d/roles/d4aa3324f0c84887b41d0dc4acf7f30e => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:41:06.912217 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f18d4d-9176-4e5b-95f2-f3be3dd57e74 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:41:06.912503 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f18d4d-9176-4e5b-95f2-f3be3dd57e74 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:41:06.912781 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f18d4d-9176-4e5b-95f2-f3be3dd57e74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:41:06.940949 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-28f18d4d-9176-4e5b-95f2-f3be3dd57e74 None None] MFA Rules not processed for user `fd3da03710e94cc7bd09a7ee7843b64d`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:41:06.943006 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-28f18d4d-9176-4e5b-95f2-f3be3dd57e74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:41:07.012264 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1158/2342] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:41:06 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:41:07.024807 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ce5b9e-9555-4319-b9f0-b70a8eb6b4ef admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:41:07.026533 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-a8ce5b9e-9555-4319-b9f0-b70a8eb6b4ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:41:07.081654 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ce5b9e-9555-4319-b9f0-b70a8eb6b4ef admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:41:07.085982 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfa2c6b-b72e-4f28-a755-7a36efe38e44 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd49388facdf446885e55b45ef91f01e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:41:07.086634 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfa2c6b-b72e-4f28-a755-7a36efe38e44 service cinder] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:41:07.086913 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfa2c6b-b72e-4f28-a755-7a36efe38e44 service cinder] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:41:07.087182 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfa2c6b-b72e-4f28-a755-7a36efe38e44 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:41:07.096486 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfa2c6b-b72e-4f28-a755-7a36efe38e44 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:41:07.097978 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfa2c6b-b72e-4f28-a755-7a36efe38e44 service cinder] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:41:07.107835 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1185/2343] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:41:07 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:41:07.357561 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f18d4d-9176-4e5b-95f2-f3be3dd57e74 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:41:07.368241 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f18d4d-9176-4e5b-95f2-f3be3dd57e74 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:41:07.372014 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0974cf-4fcf-4ae7-b4ef-baa1d9645678 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd49388facdf446885e55b45ef91f01e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:41:07.372654 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0974cf-4fcf-4ae7-b4ef-baa1d9645678 service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:41:07.372941 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0974cf-4fcf-4ae7-b4ef-baa1d9645678 service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:41:07.373219 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0974cf-4fcf-4ae7-b4ef-baa1d9645678 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:41:07.379364 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:41:07.432687 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0974cf-4fcf-4ae7-b4ef-baa1d9645678 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:41:07.434143 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0974cf-4fcf-4ae7-b4ef-baa1d9645678 service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:41:07.443002 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1159/2344] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:41:07 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:41:07.703869 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9df581-4da3-4b7f-8cb1-8824d3d56fb7 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:41:07.704374 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9df581-4da3-4b7f-8cb1-8824d3d56fb7 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:41:07.704474 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9df581-4da3-4b7f-8cb1-8824d3d56fb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:41:07.721967 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-0f9df581-4da3-4b7f-8cb1-8824d3d56fb7 None None] MFA Rules not processed for user `768f872096584d4093fc616d547aabb6`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:41:07.723185 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-0f9df581-4da3-4b7f-8cb1-8824d3d56fb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:41:07.779484 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1186/2345] 10.222.0.66 () {58 vars in 1009 bytes} [Mon Jul 18 02:41:07 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:41:07.945434 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0974cf-4fcf-4ae7-b4ef-baa1d9645678 service cinder] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:41:07.946997 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-af0974cf-4fcf-4ae7-b4ef-baa1d9645678 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:41:08.028737 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0974cf-4fcf-4ae7-b4ef-baa1d9645678 service cinder] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:41:08.033820 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2479603-2422-4116-b4ac-c8baa612f4e8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4300622cbd347dd8f5227bb960e98d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:41:08.034497 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2479603-2422-4116-b4ac-c8baa612f4e8 service swift] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:41:08.035146 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2479603-2422-4116-b4ac-c8baa612f4e8 service swift] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:41:08.035146 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2479603-2422-4116-b4ac-c8baa612f4e8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:41:08.045479 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2479603-2422-4116-b4ac-c8baa612f4e8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:41:08.047221 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2479603-2422-4116-b4ac-c8baa612f4e8 service swift] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:41:08.057599 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1160/2346] 10.222.0.66 () {60 vars in 1384 bytes} [Mon Jul 18 02:41:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 02:41:17.773049 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=73,keystone:INSERT=5 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:41:18.054352 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=56 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:41:21.667925 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1238d6c-cb00-4f87-b8d2-b24d00c5f712 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:41:21.668379 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1238d6c-cb00-4f87-b8d2-b24d00c5f712 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:41:21.668479 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1238d6c-cb00-4f87-b8d2-b24d00c5f712 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:41:21.687874 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-d1238d6c-cb00-4f87-b8d2-b24d00c5f712 None None] MFA Rules not processed for user `41a452eca3af46b09f56a80fa8652247`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:41:21.689065 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-d1238d6c-cb00-4f87-b8d2-b24d00c5f712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:41:21.742887 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1187/2347] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:41:21 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:41:26.453088 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd30fc7a-2721-4866-bc90-a6c05117ead0 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:41:26.453664 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd30fc7a-2721-4866-bc90-a6c05117ead0 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:41:26.453664 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd30fc7a-2721-4866-bc90-a6c05117ead0 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:41:26.455751 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1161/2348] 10.222.0.66 () {54 vars in 921 bytes} [Mon Jul 18 02:41:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:41:26.463635 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360d9027-f449-4231-8ae2-dd896dab24e4 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:41:26.464582 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360d9027-f449-4231-8ae2-dd896dab24e4 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:41:26.465056 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360d9027-f449-4231-8ae2-dd896dab24e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:41:26.484915 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-360d9027-f449-4231-8ae2-dd896dab24e4 None None] MFA Rules not processed for user `80b138831bc54c7b986d50092d62d4af`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:41:26.486373 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-360d9027-f449-4231-8ae2-dd896dab24e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:41:26.581945 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1188/2349] 10.222.0.66 () {58 vars in 1018 bytes} [Mon Jul 18 02:41:26 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:41:26.595023 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd30fc7a-2721-4866-bc90-a6c05117ead0 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:41:26.596554 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-cd30fc7a-2721-4866-bc90-a6c05117ead0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:41:26.666794 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd30fc7a-2721-4866-bc90-a6c05117ead0 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:41:26.670434 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8beedc-a49e-44e7-95a7-b1bd2ae05948 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1391b81e7234bc18e22b9e84f231256', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:41:26.671087 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8beedc-a49e-44e7-95a7-b1bd2ae05948 service glance] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:41:26.671371 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8beedc-a49e-44e7-95a7-b1bd2ae05948 service glance] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:41:26.671682 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8beedc-a49e-44e7-95a7-b1bd2ae05948 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:41:26.684017 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8beedc-a49e-44e7-95a7-b1bd2ae05948 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:41:26.685461 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8beedc-a49e-44e7-95a7-b1bd2ae05948 service glance] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:41:26.696948 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1162/2350] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:41:26 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 02:41:26.990550 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8e2be4-644f-44bb-b805-f048c7831d2b None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:41:26.990843 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8e2be4-644f-44bb-b805-f048c7831d2b None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:41:26.991124 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8e2be4-644f-44bb-b805-f048c7831d2b None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:41:26.992637 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1189/2351] 10.222.0.66 () {54 vars in 921 bytes} [Mon Jul 18 02:41:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:41:26.997517 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c79e28-bec0-4eb7-b774-e0810c93b884 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:41:26.997876 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c79e28-bec0-4eb7-b774-e0810c93b884 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:41:26.998110 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c79e28-bec0-4eb7-b774-e0810c93b884 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:41:27.016763 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-95c79e28-bec0-4eb7-b774-e0810c93b884 None None] MFA Rules not processed for user `eaf19a259c2741cea375a38763e33cdc`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:41:27.018031 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-95c79e28-bec0-4eb7-b774-e0810c93b884 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:41:27.074905 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1163/2352] 10.222.0.66 () {58 vars in 1018 bytes} [Mon Jul 18 02:41:26 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:41:27.088346 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8e2be4-644f-44bb-b805-f048c7831d2b None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:41:27.090346 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-1b8e2be4-644f-44bb-b805-f048c7831d2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:41:27.106487 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8e2be4-644f-44bb-b805-f048c7831d2b None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:41:27.110180 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ac4d8b-9e0e-4882-858c-929b6dcfdbda service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eaf19a259c2741cea375a38763e33cdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:41:27.110872 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ac4d8b-9e0e-4882-858c-929b6dcfdbda service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:41:27.111154 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ac4d8b-9e0e-4882-858c-929b6dcfdbda service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:41:27.111430 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ac4d8b-9e0e-4882-858c-929b6dcfdbda service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:41:27.120140 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ac4d8b-9e0e-4882-858c-929b6dcfdbda service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:41:27.121724 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ac4d8b-9e0e-4882-858c-929b6dcfdbda service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:41:27.130861 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1190/2353] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:41:27 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:41:27.251750 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b61db0-aac4-4c4f-bf25-6373911a1889 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:41:27.252153 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b61db0-aac4-4c4f-bf25-6373911a1889 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:41:27.252529 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b61db0-aac4-4c4f-bf25-6373911a1889 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:41:27.254795 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1164/2354] 10.222.0.66 () {54 vars in 921 bytes} [Mon Jul 18 02:41:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 02:41:27.262017 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427e3663-d0b4-4f43-ae5d-12e02309a49b None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:41:27.262700 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427e3663-d0b4-4f43-ae5d-12e02309a49b None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:41:27.262700 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427e3663-d0b4-4f43-ae5d-12e02309a49b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:41:27.281714 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-427e3663-d0b4-4f43-ae5d-12e02309a49b None None] MFA Rules not processed for user `80b138831bc54c7b986d50092d62d4af`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:41:27.282955 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-427e3663-d0b4-4f43-ae5d-12e02309a49b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:41:27.292889 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1191/2355] 10.222.0.66 () {58 vars in 1018 bytes} [Mon Jul 18 02:41:27 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:41:27.303563 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b61db0-aac4-4c4f-bf25-6373911a1889 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:41:27.315868 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b61db0-aac4-4c4f-bf25-6373911a1889 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:41:27.319447 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316d977e-c09d-478d-8e5e-eb12e77840ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eaf19a259c2741cea375a38763e33cdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:41:27.320092 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316d977e-c09d-478d-8e5e-eb12e77840ae service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:41:27.320407 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316d977e-c09d-478d-8e5e-eb12e77840ae service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:41:27.323330 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316d977e-c09d-478d-8e5e-eb12e77840ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:41:27.331295 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316d977e-c09d-478d-8e5e-eb12e77840ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:41:27.332724 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316d977e-c09d-478d-8e5e-eb12e77840ae service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:41:27.344241 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1165/2356] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:41:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:41:30.264030 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421c39f9-e8fa-4eab-ab5a-37989f75881c None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:41:30.264498 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421c39f9-e8fa-4eab-ab5a-37989f75881c None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:41:30.264593 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421c39f9-e8fa-4eab-ab5a-37989f75881c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:41:30.282156 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-421c39f9-e8fa-4eab-ab5a-37989f75881c None None] MFA Rules not processed for user `768f872096584d4093fc616d547aabb6`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:41:30.283403 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-421c39f9-e8fa-4eab-ab5a-37989f75881c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:41:30.291597 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1192/2357] 10.222.0.66 () {58 vars in 1009 bytes} [Mon Jul 18 02:41:30 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:41:30.303378 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316d977e-c09d-478d-8e5e-eb12e77840ae service neutron] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:41:30.315460 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316d977e-c09d-478d-8e5e-eb12e77840ae service neutron] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:41:30.320400 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c80f640-7b3e-4f68-ac2d-4c4e1fc23f2b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4300622cbd347dd8f5227bb960e98d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:41:30.321167 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c80f640-7b3e-4f68-ac2d-4c4e1fc23f2b service swift] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:41:30.321449 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c80f640-7b3e-4f68-ac2d-4c4e1fc23f2b service swift] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:41:30.321727 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c80f640-7b3e-4f68-ac2d-4c4e1fc23f2b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:41:30.330007 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c80f640-7b3e-4f68-ac2d-4c4e1fc23f2b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:41:30.331625 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c80f640-7b3e-4f68-ac2d-4c4e1fc23f2b service swift] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:41:30.340036 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1166/2358] 10.222.0.66 () {60 vars in 1384 bytes} [Mon Jul 18 02:41:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 02:41:40.276810 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=68 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:41:40.336471 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=61 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:41:47.866051 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1350fdf6-65ca-45d4-a4bb-97b53f3c23ef None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:41:47.866560 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1350fdf6-65ca-45d4-a4bb-97b53f3c23ef None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:41:47.866677 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1350fdf6-65ca-45d4-a4bb-97b53f3c23ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:41:47.883954 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-1350fdf6-65ca-45d4-a4bb-97b53f3c23ef None None] MFA Rules not processed for user `768f872096584d4093fc616d547aabb6`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:41:47.885707 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-1350fdf6-65ca-45d4-a4bb-97b53f3c23ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:41:47.893246 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1193/2359] 10.222.0.66 () {58 vars in 967 bytes} [Mon Jul 18 02:41:47 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:41:47.905714 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c80f640-7b3e-4f68-ac2d-4c4e1fc23f2b service swift] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:41:47.916249 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c80f640-7b3e-4f68-ac2d-4c4e1fc23f2b service swift] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:41:47.921270 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdc5ea5-e235-41ea-988b-9870fa816fb3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4300622cbd347dd8f5227bb960e98d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:41:47.921910 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdc5ea5-e235-41ea-988b-9870fa816fb3 service swift] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:41:47.922228 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdc5ea5-e235-41ea-988b-9870fa816fb3 service swift] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:41:47.922511 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdc5ea5-e235-41ea-988b-9870fa816fb3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:41:47.930856 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdc5ea5-e235-41ea-988b-9870fa816fb3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:41:47.932469 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdc5ea5-e235-41ea-988b-9870fa816fb3 service swift] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:41:47.940785 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1167/2360] 10.222.0.66 () {60 vars in 1384 bytes} [Mon Jul 18 02:41:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 02:41:57.878942 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=3 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:41:57.937574 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=6 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:42:00.717914 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1350fdf6-65ca-45d4-a4bb-97b53f3c23ef None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:00.783073 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1350fdf6-65ca-45d4-a4bb-97b53f3c23ef None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:00.787625 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6646beb9-1f62-4a23-a281-a17c4aa904b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:00.788488 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6646beb9-1f62-4a23-a281-a17c4aa904b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:00.788776 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6646beb9-1f62-4a23-a281-a17c4aa904b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:00.789055 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6646beb9-1f62-4a23-a281-a17c4aa904b6 admin admin] PATH_INFO: `/v3/users/57c838448349443aa08d6dc0c661584d` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:00.790275 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6646beb9-1f62-4a23-a281-a17c4aa904b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=57c838448349443aa08d6dc0c661584d)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:00.792380 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6646beb9-1f62-4a23-a281-a17c4aa904b6 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:01.051792 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-6646beb9-1f62-4a23-a281-a17c4aa904b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57c838448349443aa08d6dc0c661584d'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:01.148186 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-6646beb9-1f62-4a23-a281-a17c4aa904b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57c838448349443aa08d6dc0c661584d'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:01.156620 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-6646beb9-1f62-4a23-a281-a17c4aa904b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57c838448349443aa08d6dc0c661584d'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:01.157369 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-6646beb9-1f62-4a23-a281-a17c4aa904b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57c838448349443aa08d6dc0c661584d'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:01.170172 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-6646beb9-1f62-4a23-a281-a17c4aa904b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57c838448349443aa08d6dc0c661584d'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:01.176559 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1194/2361] 10.222.0.66 () {58 vars in 1223 bytes} [Mon Jul 18 02:42:00 2022] DELETE /identity/v3/users/57c838448349443aa08d6dc0c661584d => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:42:01.181434 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71c1d94-a773-48b3-ae65-bbb55773681f None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:01.181943 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71c1d94-a773-48b3-ae65-bbb55773681f None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:01.182043 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71c1d94-a773-48b3-ae65-bbb55773681f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:01.201195 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-b71c1d94-a773-48b3-ae65-bbb55773681f None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:42:01.202490 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-b71c1d94-a773-48b3-ae65-bbb55773681f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:01.263468 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1168/2362] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:42:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:42:01.273156 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6646beb9-1f62-4a23-a281-a17c4aa904b6 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:01.274738 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-6646beb9-1f62-4a23-a281-a17c4aa904b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:01.337143 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6646beb9-1f62-4a23-a281-a17c4aa904b6 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:01.341409 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b17e3c6-f8ef-451a-9061-f4675f9658a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eaf19a259c2741cea375a38763e33cdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:01.342061 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b17e3c6-f8ef-451a-9061-f4675f9658a8 service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:01.342384 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b17e3c6-f8ef-451a-9061-f4675f9658a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:01.342662 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b17e3c6-f8ef-451a-9061-f4675f9658a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:01.352264 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b17e3c6-f8ef-451a-9061-f4675f9658a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:01.353724 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b17e3c6-f8ef-451a-9061-f4675f9658a8 service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:01.364210 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1195/2363] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:42:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:42:02.054945 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71c1d94-a773-48b3-ae65-bbb55773681f None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:02.061018 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-b71c1d94-a773-48b3-ae65-bbb55773681f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:02.081723 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71c1d94-a773-48b3-ae65-bbb55773681f None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:02.087332 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb18a77-9df7-4f38-983a-c40292de784e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:02.088317 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb18a77-9df7-4f38-983a-c40292de784e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:02.089227 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb18a77-9df7-4f38-983a-c40292de784e admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:02.089991 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb18a77-9df7-4f38-983a-c40292de784e admin admin] PATH_INFO: `/v3/projects/8a7d4a7c8a2f4e819675f0800fa3bf40` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:02.093843 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb18a77-9df7-4f38-983a-c40292de784e admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a7d4a7c8a2f4e819675f0800fa3bf40)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:02.098269 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb18a77-9df7-4f38-983a-c40292de784e admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:02.251030 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-8bb18a77-9df7-4f38-983a-c40292de784e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a7d4a7c8a2f4e819675f0800fa3bf40', 'request_id': 'req-8bb18a77-9df7-4f38-983a-c40292de784e'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:02.257863 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-8bb18a77-9df7-4f38-983a-c40292de784e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a7d4a7c8a2f4e819675f0800fa3bf40', 'request_id': 'req-8bb18a77-9df7-4f38-983a-c40292de784e'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:02.264680 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-8bb18a77-9df7-4f38-983a-c40292de784e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a7d4a7c8a2f4e819675f0800fa3bf40', 'request_id': 'req-8bb18a77-9df7-4f38-983a-c40292de784e'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:02.360181 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-8bb18a77-9df7-4f38-983a-c40292de784e admin admin] The token cache is being invalidate because project 8a7d4a7c8a2f4e819675f0800fa3bf40 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 02:42:02.360476 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-8bb18a77-9df7-4f38-983a-c40292de784e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:02.361177 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-8bb18a77-9df7-4f38-983a-c40292de784e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:02.363144 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1169/2364] 10.222.0.66 () {58 vars in 1232 bytes} [Mon Jul 18 02:42:02 2022] DELETE /identity/v3/projects/8a7d4a7c8a2f4e819675f0800fa3bf40 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:42:02.366885 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b17e3c6-f8ef-451a-9061-f4675f9658a8 service neutron] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:02.377276 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b17e3c6-f8ef-451a-9061-f4675f9658a8 service neutron] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:02.395929 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:42:02.436526 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2294aafa-7bc5-4636-bab4-c25a81bacc97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:02.437438 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2294aafa-7bc5-4636-bab4-c25a81bacc97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:02.437718 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2294aafa-7bc5-4636-bab4-c25a81bacc97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:02.438027 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2294aafa-7bc5-4636-bab4-c25a81bacc97 admin admin] PATH_INFO: `/v3/users/fd3da03710e94cc7bd09a7ee7843b64d` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:02.439290 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2294aafa-7bc5-4636-bab4-c25a81bacc97 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd3da03710e94cc7bd09a7ee7843b64d)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:02.441258 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2294aafa-7bc5-4636-bab4-c25a81bacc97 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:02.648875 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-2294aafa-7bc5-4636-bab4-c25a81bacc97 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd3da03710e94cc7bd09a7ee7843b64d'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:02.757626 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-2294aafa-7bc5-4636-bab4-c25a81bacc97 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd3da03710e94cc7bd09a7ee7843b64d'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:02.771557 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-2294aafa-7bc5-4636-bab4-c25a81bacc97 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd3da03710e94cc7bd09a7ee7843b64d'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:02.772272 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-2294aafa-7bc5-4636-bab4-c25a81bacc97 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd3da03710e94cc7bd09a7ee7843b64d'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:02.793426 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-2294aafa-7bc5-4636-bab4-c25a81bacc97 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd3da03710e94cc7bd09a7ee7843b64d'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:02.807469 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1196/2365] 10.222.0.66 () {58 vars in 1223 bytes} [Mon Jul 18 02:42:02 2022] DELETE /identity/v3/users/fd3da03710e94cc7bd09a7ee7843b64d => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:42:03.463897 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb18a77-9df7-4f38-983a-c40292de784e admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:03.465492 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-8bb18a77-9df7-4f38-983a-c40292de784e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:03.479179 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb18a77-9df7-4f38-983a-c40292de784e admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:03.541367 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9749b7b7-8998-41a6-80d7-f3023781ecdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:03.542212 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9749b7b7-8998-41a6-80d7-f3023781ecdb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:03.542605 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9749b7b7-8998-41a6-80d7-f3023781ecdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:03.542888 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9749b7b7-8998-41a6-80d7-f3023781ecdb admin admin] PATH_INFO: `/v3/projects/67baeb6265c54e25ae0d2aa46980f34d` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:03.544159 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9749b7b7-8998-41a6-80d7-f3023781ecdb admin admin] RBAC: Authorizing `identity:delete_project(project_id=67baeb6265c54e25ae0d2aa46980f34d)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:03.546106 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9749b7b7-8998-41a6-80d7-f3023781ecdb admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:03.646764 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:42:03.664349 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-9749b7b7-8998-41a6-80d7-f3023781ecdb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67baeb6265c54e25ae0d2aa46980f34d', 'request_id': 'req-9749b7b7-8998-41a6-80d7-f3023781ecdb'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:03.670670 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-9749b7b7-8998-41a6-80d7-f3023781ecdb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67baeb6265c54e25ae0d2aa46980f34d', 'request_id': 'req-9749b7b7-8998-41a6-80d7-f3023781ecdb'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:03.676984 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-9749b7b7-8998-41a6-80d7-f3023781ecdb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67baeb6265c54e25ae0d2aa46980f34d', 'request_id': 'req-9749b7b7-8998-41a6-80d7-f3023781ecdb'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:03.761953 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-9749b7b7-8998-41a6-80d7-f3023781ecdb admin admin] The token cache is being invalidate because project 67baeb6265c54e25ae0d2aa46980f34d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 02:42:03.764244 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-9749b7b7-8998-41a6-80d7-f3023781ecdb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:03.764244 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-9749b7b7-8998-41a6-80d7-f3023781ecdb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:03.764872 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1170/2366] 10.222.0.66 () {58 vars in 1232 bytes} [Mon Jul 18 02:42:03 2022] DELETE /identity/v3/projects/67baeb6265c54e25ae0d2aa46980f34d => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:42:03.769761 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffa2449-190c-4fc9-b21c-cbe94ed241f8 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:03.770297 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffa2449-190c-4fc9-b21c-cbe94ed241f8 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:03.770604 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffa2449-190c-4fc9-b21c-cbe94ed241f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:03.790942 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-8ffa2449-190c-4fc9-b21c-cbe94ed241f8 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:42:03.792348 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-8ffa2449-190c-4fc9-b21c-cbe94ed241f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:03.864296 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1197/2367] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:42:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:42:03.874844 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6e1981-2c29-4bf5-8388-ac282890d643 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:03.875156 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6e1981-2c29-4bf5-8388-ac282890d643 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:03.875443 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6e1981-2c29-4bf5-8388-ac282890d643 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:03.894056 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-5e6e1981-2c29-4bf5-8388-ac282890d643 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:42:03.895299 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-5e6e1981-2c29-4bf5-8388-ac282890d643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:03.903491 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1171/2368] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:42:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:42:03.907736 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffa2449-190c-4fc9-b21c-cbe94ed241f8 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:03.917314 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffa2449-190c-4fc9-b21c-cbe94ed241f8 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:03.921909 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ea21b9-a53f-42ab-9427-7d96f3076150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:03.922658 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ea21b9-a53f-42ab-9427-7d96f3076150 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:03.922945 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ea21b9-a53f-42ab-9427-7d96f3076150 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:03.923224 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ea21b9-a53f-42ab-9427-7d96f3076150 admin admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:03.923946 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ea21b9-a53f-42ab-9427-7d96f3076150 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:42:03.924254 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ea21b9-a53f-42ab-9427-7d96f3076150 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:03.925867 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ea21b9-a53f-42ab-9427-7d96f3076150 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:03.938211 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1198/2369] 10.222.0.66 () {58 vars in 1152 bytes} [Mon Jul 18 02:42:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:42:03.942496 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6e1981-2c29-4bf5-8388-ac282890d643 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:03.952457 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6e1981-2c29-4bf5-8388-ac282890d643 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:03.957463 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406b62e7-4f0e-4ef8-ac7d-e5fc5d5b197c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:03.958215 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406b62e7-4f0e-4ef8-ac7d-e5fc5d5b197c admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:03.958503 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406b62e7-4f0e-4ef8-ac7d-e5fc5d5b197c admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:03.958781 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406b62e7-4f0e-4ef8-ac7d-e5fc5d5b197c admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:03.959484 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406b62e7-4f0e-4ef8-ac7d-e5fc5d5b197c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:03.961545 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406b62e7-4f0e-4ef8-ac7d-e5fc5d5b197c admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:04.093030 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1172/2370] 10.222.0.66 () {60 vars in 1152 bytes} [Mon Jul 18 02:42:03 2022] POST /identity/v3/projects => generated 381 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:42:04.096968 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ea21b9-a53f-42ab-9427-7d96f3076150 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:04.107450 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ea21b9-a53f-42ab-9427-7d96f3076150 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:04.135386 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:42:04.171079 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50513dc-e137-4c5e-8283-7081c018ac55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:04.172123 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50513dc-e137-4c5e-8283-7081c018ac55 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:04.172703 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50513dc-e137-4c5e-8283-7081c018ac55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:04.173166 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50513dc-e137-4c5e-8283-7081c018ac55 admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:04.174574 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50513dc-e137-4c5e-8283-7081c018ac55 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:04.179007 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50513dc-e137-4c5e-8283-7081c018ac55 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:04.335239 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1199/2371] 10.222.0.66 () {60 vars in 1143 bytes} [Mon Jul 18 02:42:04 2022] POST /identity/v3/users => generated 291 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:42:04.339045 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406b62e7-4f0e-4ef8-ac7d-e5fc5d5b197c admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:04.349804 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406b62e7-4f0e-4ef8-ac7d-e5fc5d5b197c admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:04.354159 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184a1a71-ed35-43c3-87b1-2493b2476a84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:04.354819 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184a1a71-ed35-43c3-87b1-2493b2476a84 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:04.355631 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184a1a71-ed35-43c3-87b1-2493b2476a84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:04.356010 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184a1a71-ed35-43c3-87b1-2493b2476a84 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:04.356908 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184a1a71-ed35-43c3-87b1-2493b2476a84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:04.358573 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184a1a71-ed35-43c3-87b1-2493b2476a84 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:04.368892 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1173/2372] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:42:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:42:04.373349 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50513dc-e137-4c5e-8283-7081c018ac55 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:04.383243 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50513dc-e137-4c5e-8283-7081c018ac55 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:04.387984 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5994b0cc-e8be-4ef0-a13f-30fec23a967e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:04.388639 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5994b0cc-e8be-4ef0-a13f-30fec23a967e admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:04.388947 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5994b0cc-e8be-4ef0-a13f-30fec23a967e admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:04.389205 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5994b0cc-e8be-4ef0-a13f-30fec23a967e admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:04.391337 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5994b0cc-e8be-4ef0-a13f-30fec23a967e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:04.393774 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5994b0cc-e8be-4ef0-a13f-30fec23a967e admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:04.408014 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1200/2373] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:42:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:42:04.409029 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184a1a71-ed35-43c3-87b1-2493b2476a84 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:04.420035 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184a1a71-ed35-43c3-87b1-2493b2476a84 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:04.424741 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435a39fd-91fc-490b-8fa3-116d2bb232c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:04.425669 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435a39fd-91fc-490b-8fa3-116d2bb232c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:04.426070 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435a39fd-91fc-490b-8fa3-116d2bb232c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:04.426378 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435a39fd-91fc-490b-8fa3-116d2bb232c1 admin admin] PATH_INFO: `/v3/projects/33cb17c7dce94b7297462fc1f2230890/users/68e9821422de4f828046c94474a6eeb3/roles/65feabf3b27944a8b01a86c8ac608457` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:04.443043 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435a39fd-91fc-490b-8fa3-116d2bb232c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=33cb17c7dce94b7297462fc1f2230890, user_id=68e9821422de4f828046c94474a6eeb3, role_id=65feabf3b27944a8b01a86c8ac608457)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:04.447192 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435a39fd-91fc-490b-8fa3-116d2bb232c1 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:04.554564 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1174/2374] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:42:04 2022] PUT /identity/v3/projects/33cb17c7dce94b7297462fc1f2230890/users/68e9821422de4f828046c94474a6eeb3/roles/65feabf3b27944a8b01a86c8ac608457 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 02:42:04.566183 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facb2963-6d15-423b-b0d4-3c456330e4a8 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:04.566480 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facb2963-6d15-423b-b0d4-3c456330e4a8 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:04.566761 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facb2963-6d15-423b-b0d4-3c456330e4a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:04.596551 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-facb2963-6d15-423b-b0d4-3c456330e4a8 None None] MFA Rules not processed for user `68e9821422de4f828046c94474a6eeb3`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:42:04.597788 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-facb2963-6d15-423b-b0d4-3c456330e4a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:04.665005 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1201/2375] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:42:04 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:42:04.674522 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435a39fd-91fc-490b-8fa3-116d2bb232c1 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:04.739114 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435a39fd-91fc-490b-8fa3-116d2bb232c1 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:04.744098 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70783e2-f617-4de2-a7eb-805aeffb79ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:04.744786 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70783e2-f617-4de2-a7eb-805aeffb79ef admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:04.745075 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70783e2-f617-4de2-a7eb-805aeffb79ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:04.745356 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70783e2-f617-4de2-a7eb-805aeffb79ef admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:04.746191 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70783e2-f617-4de2-a7eb-805aeffb79ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:04.748170 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70783e2-f617-4de2-a7eb-805aeffb79ef admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:04.987165 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1175/2376] 10.222.0.66 () {60 vars in 1152 bytes} [Mon Jul 18 02:42:04 2022] POST /identity/v3/projects => generated 379 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 02:42:04.990832 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facb2963-6d15-423b-b0d4-3c456330e4a8 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:05.002993 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facb2963-6d15-423b-b0d4-3c456330e4a8 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:05.067348 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2761803e-9f87-4ea8-b425-121b69fcd458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:05.068018 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2761803e-9f87-4ea8-b425-121b69fcd458 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:05.068305 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2761803e-9f87-4ea8-b425-121b69fcd458 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:05.068583 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2761803e-9f87-4ea8-b425-121b69fcd458 admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:05.069270 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2761803e-9f87-4ea8-b425-121b69fcd458 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:05.071322 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2761803e-9f87-4ea8-b425-121b69fcd458 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:05.206628 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1202/2377] 10.222.0.66 () {60 vars in 1143 bytes} [Mon Jul 18 02:42:04 2022] POST /identity/v3/users => generated 296 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:42:05.210638 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70783e2-f617-4de2-a7eb-805aeffb79ef admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:05.221954 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70783e2-f617-4de2-a7eb-805aeffb79ef admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:05.226576 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048026a1-3813-4392-a1ea-ef947ebce9a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:05.227241 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048026a1-3813-4392-a1ea-ef947ebce9a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:05.227528 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048026a1-3813-4392-a1ea-ef947ebce9a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:05.227809 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048026a1-3813-4392-a1ea-ef947ebce9a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:05.228528 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048026a1-3813-4392-a1ea-ef947ebce9a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:05.230213 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048026a1-3813-4392-a1ea-ef947ebce9a9 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:05.241217 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1176/2378] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:42:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:42:05.245340 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2761803e-9f87-4ea8-b425-121b69fcd458 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:05.256162 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2761803e-9f87-4ea8-b425-121b69fcd458 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:05.260988 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb20969-9dcf-4d1d-aabd-cfbdbca43d36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:05.262532 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb20969-9dcf-4d1d-aabd-cfbdbca43d36 admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:05.262836 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb20969-9dcf-4d1d-aabd-cfbdbca43d36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:05.263117 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb20969-9dcf-4d1d-aabd-cfbdbca43d36 admin admin] PATH_INFO: `/v3/projects/acce66ef33b840a18bf0a49048f10a25/users/031aee7ffef54563be92690216b429b2/roles/d4aa3324f0c84887b41d0dc4acf7f30e` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:05.279279 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb20969-9dcf-4d1d-aabd-cfbdbca43d36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=acce66ef33b840a18bf0a49048f10a25, user_id=031aee7ffef54563be92690216b429b2, role_id=d4aa3324f0c84887b41d0dc4acf7f30e)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:05.282159 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb20969-9dcf-4d1d-aabd-cfbdbca43d36 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:05.353015 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1203/2379] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:42:05 2022] PUT /identity/v3/projects/acce66ef33b840a18bf0a49048f10a25/users/031aee7ffef54563be92690216b429b2/roles/d4aa3324f0c84887b41d0dc4acf7f30e => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:42:05.358672 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048026a1-3813-4392-a1ea-ef947ebce9a9 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:05.368773 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048026a1-3813-4392-a1ea-ef947ebce9a9 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:05.388994 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:42:05.426796 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4524b49-7afd-449c-b466-60bf32bfa9e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:05.428627 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4524b49-7afd-449c-b466-60bf32bfa9e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:05.428627 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4524b49-7afd-449c-b466-60bf32bfa9e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:05.428627 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4524b49-7afd-449c-b466-60bf32bfa9e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:05.428886 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4524b49-7afd-449c-b466-60bf32bfa9e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:05.430402 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4524b49-7afd-449c-b466-60bf32bfa9e1 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:05.440972 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1177/2380] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:42:05 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:42:05.445168 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb20969-9dcf-4d1d-aabd-cfbdbca43d36 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:05.450895 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:42:05.456283 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb20969-9dcf-4d1d-aabd-cfbdbca43d36 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:05.461788 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff84b47-e121-4373-b8a7-1500edc7a059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:05.462779 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff84b47-e121-4373-b8a7-1500edc7a059 admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:05.463060 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff84b47-e121-4373-b8a7-1500edc7a059 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:05.463332 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff84b47-e121-4373-b8a7-1500edc7a059 admin admin] PATH_INFO: `/v3/projects/acce66ef33b840a18bf0a49048f10a25/users/031aee7ffef54563be92690216b429b2/roles/d4aa3324f0c84887b41d0dc4acf7f30e` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:05.465981 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff84b47-e121-4373-b8a7-1500edc7a059 admin admin] RBAC: Authorizing `identity:create_grant(project_id=acce66ef33b840a18bf0a49048f10a25, user_id=031aee7ffef54563be92690216b429b2, role_id=d4aa3324f0c84887b41d0dc4acf7f30e)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:05.468747 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff84b47-e121-4373-b8a7-1500edc7a059 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:05.479374 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1204/2381] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:42:05 2022] PUT /identity/v3/projects/acce66ef33b840a18bf0a49048f10a25/users/031aee7ffef54563be92690216b429b2/roles/d4aa3324f0c84887b41d0dc4acf7f30e => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:42:05.490469 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe98ded-db23-4bed-9dd1-f3648f0a0a3f None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:05.490765 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe98ded-db23-4bed-9dd1-f3648f0a0a3f None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:05.491048 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe98ded-db23-4bed-9dd1-f3648f0a0a3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:05.516937 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-dfe98ded-db23-4bed-9dd1-f3648f0a0a3f None None] MFA Rules not processed for user `031aee7ffef54563be92690216b429b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:42:05.518177 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-dfe98ded-db23-4bed-9dd1-f3648f0a0a3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:05.588081 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1178/2382] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:42:05 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:42:05.599924 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff84b47-e121-4373-b8a7-1500edc7a059 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:05.601626 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-cff84b47-e121-4373-b8a7-1500edc7a059 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:05.662114 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff84b47-e121-4373-b8a7-1500edc7a059 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:05.665991 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6e19f5-da0e-406f-bfcd-44ef2a31c1ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd49388facdf446885e55b45ef91f01e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:05.666784 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6e19f5-da0e-406f-bfcd-44ef2a31c1ca service cinder] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:05.667235 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6e19f5-da0e-406f-bfcd-44ef2a31c1ca service cinder] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:05.667940 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6e19f5-da0e-406f-bfcd-44ef2a31c1ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:05.677436 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6e19f5-da0e-406f-bfcd-44ef2a31c1ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:05.678898 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6e19f5-da0e-406f-bfcd-44ef2a31c1ca service cinder] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:05.688837 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1205/2383] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:42:05 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:42:05.944927 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe98ded-db23-4bed-9dd1-f3648f0a0a3f None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:05.955224 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe98ded-db23-4bed-9dd1-f3648f0a0a3f None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:05.958808 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9ae153-3f52-4a33-9d4a-45769b98f540 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd49388facdf446885e55b45ef91f01e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:05.959450 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9ae153-3f52-4a33-9d4a-45769b98f540 service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:05.959733 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9ae153-3f52-4a33-9d4a-45769b98f540 service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:05.960010 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9ae153-3f52-4a33-9d4a-45769b98f540 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:06.017065 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9ae153-3f52-4a33-9d4a-45769b98f540 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:06.018567 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9ae153-3f52-4a33-9d4a-45769b98f540 service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:06.028330 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1179/2384] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:42:05 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:42:15.937195 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=30,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:42:16.057298 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=80 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:42:17.126185 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6e19f5-da0e-406f-bfcd-44ef2a31c1ca service cinder] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:17.127959 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-fe6e19f5-da0e-406f-bfcd-44ef2a31c1ca service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:17.186745 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6e19f5-da0e-406f-bfcd-44ef2a31c1ca service cinder] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:17.190591 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67d309f-a39b-4c5d-b11a-64ffde1e40ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eaf19a259c2741cea375a38763e33cdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:17.191295 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67d309f-a39b-4c5d-b11a-64ffde1e40ba service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:17.191583 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67d309f-a39b-4c5d-b11a-64ffde1e40ba service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:17.191861 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67d309f-a39b-4c5d-b11a-64ffde1e40ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:17.193721 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-f67d309f-a39b-4c5d-b11a-64ffde1e40ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:17.259735 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67d309f-a39b-4c5d-b11a-64ffde1e40ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:17.261191 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67d309f-a39b-4c5d-b11a-64ffde1e40ba service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:17.272046 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1206/2385] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:42:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 02:42:18.172621 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9ae153-3f52-4a33-9d4a-45769b98f540 service cinder] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:18.196274 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9ae153-3f52-4a33-9d4a-45769b98f540 service cinder] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:18.205487 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66473177-c466-430e-906b-99d3726ffe2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:18.207163 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66473177-c466-430e-906b-99d3726ffe2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:18.207163 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66473177-c466-430e-906b-99d3726ffe2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:18.207163 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66473177-c466-430e-906b-99d3726ffe2c admin admin] PATH_INFO: `/v3/users/68e9821422de4f828046c94474a6eeb3` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:18.208746 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66473177-c466-430e-906b-99d3726ffe2c admin admin] RBAC: Authorizing `identity:delete_user(user_id=68e9821422de4f828046c94474a6eeb3)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:18.210841 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66473177-c466-430e-906b-99d3726ffe2c admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:18.534109 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-66473177-c466-430e-906b-99d3726ffe2c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '68e9821422de4f828046c94474a6eeb3'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:18.622252 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-66473177-c466-430e-906b-99d3726ffe2c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '68e9821422de4f828046c94474a6eeb3'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:18.623114 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-66473177-c466-430e-906b-99d3726ffe2c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '68e9821422de4f828046c94474a6eeb3'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:18.640334 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-66473177-c466-430e-906b-99d3726ffe2c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '68e9821422de4f828046c94474a6eeb3'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:18.641320 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-66473177-c466-430e-906b-99d3726ffe2c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '68e9821422de4f828046c94474a6eeb3'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:18.656923 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1180/2386] 10.222.0.66 () {58 vars in 1223 bytes} [Mon Jul 18 02:42:18 2022] DELETE /identity/v3/users/68e9821422de4f828046c94474a6eeb3 => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 02:42:18.662356 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb21ca5-ddf1-4fe9-89c8-94db1f20ec6c None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:18.662989 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb21ca5-ddf1-4fe9-89c8-94db1f20ec6c None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:18.662989 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb21ca5-ddf1-4fe9-89c8-94db1f20ec6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:18.691708 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-abb21ca5-ddf1-4fe9-89c8-94db1f20ec6c None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:42:18.693358 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-abb21ca5-ddf1-4fe9-89c8-94db1f20ec6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:18.763055 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1207/2387] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:42:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:42:18.773412 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66473177-c466-430e-906b-99d3726ffe2c admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:18.775591 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-66473177-c466-430e-906b-99d3726ffe2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:18.861486 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66473177-c466-430e-906b-99d3726ffe2c admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:18.865985 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f267ecad-34ee-426d-8b77-57eeeb515524 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eaf19a259c2741cea375a38763e33cdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:18.866839 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f267ecad-34ee-426d-8b77-57eeeb515524 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:18.867059 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f267ecad-34ee-426d-8b77-57eeeb515524 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:18.867251 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f267ecad-34ee-426d-8b77-57eeeb515524 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:18.878817 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f267ecad-34ee-426d-8b77-57eeeb515524 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:18.881342 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f267ecad-34ee-426d-8b77-57eeeb515524 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:18.893045 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1181/2388] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:42:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:42:19.521339 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb21ca5-ddf1-4fe9-89c8-94db1f20ec6c None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:19.523651 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-abb21ca5-ddf1-4fe9-89c8-94db1f20ec6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:19.550484 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb21ca5-ddf1-4fe9-89c8-94db1f20ec6c None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:19.567269 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4769278c-f4a6-4da5-bd31-6c3fb6b51848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:19.568401 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4769278c-f4a6-4da5-bd31-6c3fb6b51848 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:19.568944 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4769278c-f4a6-4da5-bd31-6c3fb6b51848 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:19.569903 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4769278c-f4a6-4da5-bd31-6c3fb6b51848 admin admin] PATH_INFO: `/v3/projects/33cb17c7dce94b7297462fc1f2230890` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:19.571654 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4769278c-f4a6-4da5-bd31-6c3fb6b51848 admin admin] RBAC: Authorizing `identity:delete_project(project_id=33cb17c7dce94b7297462fc1f2230890)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:19.575159 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4769278c-f4a6-4da5-bd31-6c3fb6b51848 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:19.695358 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:42:19.698366 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-4769278c-f4a6-4da5-bd31-6c3fb6b51848 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33cb17c7dce94b7297462fc1f2230890', 'request_id': 'req-4769278c-f4a6-4da5-bd31-6c3fb6b51848'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:19.703934 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-4769278c-f4a6-4da5-bd31-6c3fb6b51848 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33cb17c7dce94b7297462fc1f2230890', 'request_id': 'req-4769278c-f4a6-4da5-bd31-6c3fb6b51848'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:19.804326 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-4769278c-f4a6-4da5-bd31-6c3fb6b51848 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33cb17c7dce94b7297462fc1f2230890', 'request_id': 'req-4769278c-f4a6-4da5-bd31-6c3fb6b51848'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:19.823633 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-4769278c-f4a6-4da5-bd31-6c3fb6b51848 admin admin] The token cache is being invalidate because project 33cb17c7dce94b7297462fc1f2230890 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 02:42:19.824098 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-4769278c-f4a6-4da5-bd31-6c3fb6b51848 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:19.825055 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-4769278c-f4a6-4da5-bd31-6c3fb6b51848 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:19.827109 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1208/2389] 10.222.0.66 () {58 vars in 1232 bytes} [Mon Jul 18 02:42:19 2022] DELETE /identity/v3/projects/33cb17c7dce94b7297462fc1f2230890 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 02:42:19.831289 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f267ecad-34ee-426d-8b77-57eeeb515524 service neutron] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:19.844504 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f267ecad-34ee-426d-8b77-57eeeb515524 service neutron] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:19.943972 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae710ff1-b5d4-4bf9-9251-f0eee5c7a337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:19.946620 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae710ff1-b5d4-4bf9-9251-f0eee5c7a337 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:19.946620 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae710ff1-b5d4-4bf9-9251-f0eee5c7a337 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:19.946620 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae710ff1-b5d4-4bf9-9251-f0eee5c7a337 admin admin] PATH_INFO: `/v3/users/031aee7ffef54563be92690216b429b2` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:19.947014 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae710ff1-b5d4-4bf9-9251-f0eee5c7a337 admin admin] RBAC: Authorizing `identity:delete_user(user_id=031aee7ffef54563be92690216b429b2)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:19.949679 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae710ff1-b5d4-4bf9-9251-f0eee5c7a337 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:19.974055 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:42:20.278966 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-ae710ff1-b5d4-4bf9-9251-f0eee5c7a337 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '031aee7ffef54563be92690216b429b2'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:20.423441 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-ae710ff1-b5d4-4bf9-9251-f0eee5c7a337 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '031aee7ffef54563be92690216b429b2'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:20.424445 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-ae710ff1-b5d4-4bf9-9251-f0eee5c7a337 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '031aee7ffef54563be92690216b429b2'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:20.446059 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-ae710ff1-b5d4-4bf9-9251-f0eee5c7a337 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '031aee7ffef54563be92690216b429b2'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:20.446933 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-ae710ff1-b5d4-4bf9-9251-f0eee5c7a337 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '031aee7ffef54563be92690216b429b2'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:20.462013 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1182/2390] 10.222.0.66 () {58 vars in 1223 bytes} [Mon Jul 18 02:42:19 2022] DELETE /identity/v3/users/031aee7ffef54563be92690216b429b2 => generated 0 bytes in 632 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:42:21.037010 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4769278c-f4a6-4da5-bd31-6c3fb6b51848 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:21.039678 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-4769278c-f4a6-4da5-bd31-6c3fb6b51848 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:21.059164 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4769278c-f4a6-4da5-bd31-6c3fb6b51848 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:21.147291 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a489d9-b0ac-4739-bd6c-28ee9876260c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:21.148178 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a489d9-b0ac-4739-bd6c-28ee9876260c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:21.148459 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a489d9-b0ac-4739-bd6c-28ee9876260c admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:21.155235 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a489d9-b0ac-4739-bd6c-28ee9876260c admin admin] PATH_INFO: `/v3/projects/acce66ef33b840a18bf0a49048f10a25` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:21.155235 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a489d9-b0ac-4739-bd6c-28ee9876260c admin admin] RBAC: Authorizing `identity:delete_project(project_id=acce66ef33b840a18bf0a49048f10a25)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:21.155235 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a489d9-b0ac-4739-bd6c-28ee9876260c admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:21.329903 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-98a489d9-b0ac-4739-bd6c-28ee9876260c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'acce66ef33b840a18bf0a49048f10a25', 'request_id': 'req-98a489d9-b0ac-4739-bd6c-28ee9876260c'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:21.335262 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-98a489d9-b0ac-4739-bd6c-28ee9876260c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'acce66ef33b840a18bf0a49048f10a25', 'request_id': 'req-98a489d9-b0ac-4739-bd6c-28ee9876260c'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:21.432505 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-98a489d9-b0ac-4739-bd6c-28ee9876260c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'acce66ef33b840a18bf0a49048f10a25', 'request_id': 'req-98a489d9-b0ac-4739-bd6c-28ee9876260c'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:21.444511 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-98a489d9-b0ac-4739-bd6c-28ee9876260c admin admin] The token cache is being invalidate because project acce66ef33b840a18bf0a49048f10a25 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 02:42:21.444900 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-98a489d9-b0ac-4739-bd6c-28ee9876260c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:21.445593 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-98a489d9-b0ac-4739-bd6c-28ee9876260c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:21.447238 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1209/2391] 10.222.0.66 () {58 vars in 1232 bytes} [Mon Jul 18 02:42:21 2022] DELETE /identity/v3/projects/acce66ef33b840a18bf0a49048f10a25 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:42:21.453140 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba36037-d4f0-4d85-a938-82f30c9e77e0 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:21.453656 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba36037-d4f0-4d85-a938-82f30c9e77e0 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:21.453772 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba36037-d4f0-4d85-a938-82f30c9e77e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:21.473777 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-cba36037-d4f0-4d85-a938-82f30c9e77e0 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:42:21.475065 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-cba36037-d4f0-4d85-a938-82f30c9e77e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:21.540082 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1183/2392] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:42:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:42:21.548364 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5eca2c-380c-4711-9410-6b4e0839635c None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:21.548653 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5eca2c-380c-4711-9410-6b4e0839635c None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:21.548933 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5eca2c-380c-4711-9410-6b4e0839635c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:21.579406 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-3b5eca2c-380c-4711-9410-6b4e0839635c None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:42:21.580633 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-3b5eca2c-380c-4711-9410-6b4e0839635c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:21.589396 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1210/2393] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:42:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:42:21.602617 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba36037-d4f0-4d85-a938-82f30c9e77e0 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:21.613563 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba36037-d4f0-4d85-a938-82f30c9e77e0 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:21.618299 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91617ce9-2daf-4ea3-9fab-22e269f3e257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:21.618964 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91617ce9-2daf-4ea3-9fab-22e269f3e257 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:21.619367 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91617ce9-2daf-4ea3-9fab-22e269f3e257 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:21.619655 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91617ce9-2daf-4ea3-9fab-22e269f3e257 admin admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:21.620427 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91617ce9-2daf-4ea3-9fab-22e269f3e257 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:42:21.620722 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91617ce9-2daf-4ea3-9fab-22e269f3e257 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:21.622614 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91617ce9-2daf-4ea3-9fab-22e269f3e257 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:21.635125 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1184/2394] 10.222.0.66 () {58 vars in 1152 bytes} [Mon Jul 18 02:42:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:42:21.639834 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5eca2c-380c-4711-9410-6b4e0839635c None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:21.649141 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5eca2c-380c-4711-9410-6b4e0839635c None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:21.654453 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1513285-63b6-46aa-9580-49f5cf65817a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:21.655105 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1513285-63b6-46aa-9580-49f5cf65817a admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:21.655394 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1513285-63b6-46aa-9580-49f5cf65817a admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:21.655670 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1513285-63b6-46aa-9580-49f5cf65817a admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:21.656393 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1513285-63b6-46aa-9580-49f5cf65817a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:21.658377 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1513285-63b6-46aa-9580-49f5cf65817a admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:21.713261 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91617ce9-2daf-4ea3-9fab-22e269f3e257 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:21.716120 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-91617ce9-2daf-4ea3-9fab-22e269f3e257 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:21.732227 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91617ce9-2daf-4ea3-9fab-22e269f3e257 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:21.737482 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95175833-b3a0-4653-964b-471eca1aa583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:21.738639 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95175833-b3a0-4653-964b-471eca1aa583 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:21.738927 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95175833-b3a0-4653-964b-471eca1aa583 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:21.739211 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95175833-b3a0-4653-964b-471eca1aa583 admin admin] PATH_INFO: `/v3/users/94c16391d39f4498a55c89bf2838788f` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:21.740444 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95175833-b3a0-4653-964b-471eca1aa583 admin admin] RBAC: Authorizing `identity:delete_user(user_id=94c16391d39f4498a55c89bf2838788f)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:21.742478 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95175833-b3a0-4653-964b-471eca1aa583 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:21.796804 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1211/2395] 10.222.0.66 () {60 vars in 1152 bytes} [Mon Jul 18 02:42:21 2022] POST /identity/v3/projects => generated 381 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:42:21.800887 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1513285-63b6-46aa-9580-49f5cf65817a admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:21.842767 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:DELETE=11,keystone:SELECT=86,keystone:INSERT=3 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:42:21.865019 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1513285-63b6-46aa-9580-49f5cf65817a admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:21.875492 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971c8afd-8367-4621-8146-ce7d203e4cdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:21.876181 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971c8afd-8367-4621-8146-ce7d203e4cdf admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:21.876469 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971c8afd-8367-4621-8146-ce7d203e4cdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:21.876746 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971c8afd-8367-4621-8146-ce7d203e4cdf admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:21.877449 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971c8afd-8367-4621-8146-ce7d203e4cdf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:21.879476 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971c8afd-8367-4621-8146-ce7d203e4cdf admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:21.985713 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-95175833-b3a0-4653-964b-471eca1aa583 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '94c16391d39f4498a55c89bf2838788f'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:22.001254 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1212/2396] 10.222.0.66 () {60 vars in 1143 bytes} [Mon Jul 18 02:42:21 2022] POST /identity/v3/users => generated 291 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 02:42:22.005327 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971c8afd-8367-4621-8146-ce7d203e4cdf admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:22.015143 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971c8afd-8367-4621-8146-ce7d203e4cdf admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:22.019717 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a752d056-4f32-4fdf-a8a1-c58ec0338e72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:22.020349 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a752d056-4f32-4fdf-a8a1-c58ec0338e72 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:22.020629 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a752d056-4f32-4fdf-a8a1-c58ec0338e72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:22.020904 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a752d056-4f32-4fdf-a8a1-c58ec0338e72 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:22.021611 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a752d056-4f32-4fdf-a8a1-c58ec0338e72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:22.023285 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a752d056-4f32-4fdf-a8a1-c58ec0338e72 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:22.034322 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1213/2397] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:42:22 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 02:42:22.045877 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a752d056-4f32-4fdf-a8a1-c58ec0338e72 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:22.055218 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a752d056-4f32-4fdf-a8a1-c58ec0338e72 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:22.059952 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f86c86-80b4-4271-ab1b-18178ebab82b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:22.060566 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f86c86-80b4-4271-ab1b-18178ebab82b admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:22.060847 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f86c86-80b4-4271-ab1b-18178ebab82b admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:22.061388 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f86c86-80b4-4271-ab1b-18178ebab82b admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:22.062064 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f86c86-80b4-4271-ab1b-18178ebab82b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:22.063640 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f86c86-80b4-4271-ab1b-18178ebab82b admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:22.075183 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1214/2398] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:42:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 02:42:22.079026 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f86c86-80b4-4271-ab1b-18178ebab82b admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:22.088540 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f86c86-80b4-4271-ab1b-18178ebab82b admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:22.093866 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000b4ae0-1c80-471c-85d9-cb8d0711bd3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:22.094755 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000b4ae0-1c80-471c-85d9-cb8d0711bd3b admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:22.095043 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000b4ae0-1c80-471c-85d9-cb8d0711bd3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:22.095317 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000b4ae0-1c80-471c-85d9-cb8d0711bd3b admin admin] PATH_INFO: `/v3/projects/56b9f450bc2c4910b141e7412653df30/users/22e282b85acc4dd18c0545a42fa10d36/roles/65feabf3b27944a8b01a86c8ac608457` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:22.110241 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000b4ae0-1c80-471c-85d9-cb8d0711bd3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=56b9f450bc2c4910b141e7412653df30, user_id=22e282b85acc4dd18c0545a42fa10d36, role_id=65feabf3b27944a8b01a86c8ac608457)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:22.113035 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000b4ae0-1c80-471c-85d9-cb8d0711bd3b admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:22.119703 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-95175833-b3a0-4653-964b-471eca1aa583 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '94c16391d39f4498a55c89bf2838788f'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:22.120334 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-95175833-b3a0-4653-964b-471eca1aa583 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '94c16391d39f4498a55c89bf2838788f'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:22.132915 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-95175833-b3a0-4653-964b-471eca1aa583 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '94c16391d39f4498a55c89bf2838788f'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:22.133544 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-95175833-b3a0-4653-964b-471eca1aa583 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '94c16391d39f4498a55c89bf2838788f'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:22.138767 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:DELETE=19,keystone:SELECT=79,keystone:INSERT=2 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:42:22.146933 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1185/2399] 10.222.0.66 () {58 vars in 1223 bytes} [Mon Jul 18 02:42:21 2022] DELETE /identity/v3/users/94c16391d39f4498a55c89bf2838788f => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 02:42:22.244635 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1215/2400] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:42:22 2022] PUT /identity/v3/projects/56b9f450bc2c4910b141e7412653df30/users/22e282b85acc4dd18c0545a42fa10d36/roles/65feabf3b27944a8b01a86c8ac608457 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:42:22.256707 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ae9cea-e7c5-4447-9139-99a08fa25559 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:22.257052 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ae9cea-e7c5-4447-9139-99a08fa25559 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:22.257332 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ae9cea-e7c5-4447-9139-99a08fa25559 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:22.294193 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-48ae9cea-e7c5-4447-9139-99a08fa25559 None None] MFA Rules not processed for user `22e282b85acc4dd18c0545a42fa10d36`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:42:22.295458 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-48ae9cea-e7c5-4447-9139-99a08fa25559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:22.362011 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1186/2401] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:42:22 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:42:22.366755 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000b4ae0-1c80-471c-85d9-cb8d0711bd3b admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:22.368486 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-000b4ae0-1c80-471c-85d9-cb8d0711bd3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:22.392968 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ae9cea-e7c5-4447-9139-99a08fa25559 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:22.394425 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-48ae9cea-e7c5-4447-9139-99a08fa25559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:22.440031 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000b4ae0-1c80-471c-85d9-cb8d0711bd3b admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:22.445154 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71399b7-2a8a-4df5-bce8-1c3d1abf480a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:22.445785 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71399b7-2a8a-4df5-bce8-1c3d1abf480a admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:22.446073 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71399b7-2a8a-4df5-bce8-1c3d1abf480a admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:22.446468 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71399b7-2a8a-4df5-bce8-1c3d1abf480a admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:22.447217 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71399b7-2a8a-4df5-bce8-1c3d1abf480a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:22.449302 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71399b7-2a8a-4df5-bce8-1c3d1abf480a admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:22.473678 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ae9cea-e7c5-4447-9139-99a08fa25559 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:22.478545 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026a1779-c6e0-49fa-8362-b3bd9258335d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:22.479411 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026a1779-c6e0-49fa-8362-b3bd9258335d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:22.479729 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026a1779-c6e0-49fa-8362-b3bd9258335d admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:22.480013 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026a1779-c6e0-49fa-8362-b3bd9258335d admin admin] PATH_INFO: `/v3/projects/4b02a417904a474abcab1eec5cea9c7e` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:22.481989 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026a1779-c6e0-49fa-8362-b3bd9258335d admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b02a417904a474abcab1eec5cea9c7e)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:22.484093 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026a1779-c6e0-49fa-8362-b3bd9258335d admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:22.567429 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1216/2402] 10.222.0.66 () {60 vars in 1152 bytes} [Mon Jul 18 02:42:22 2022] POST /identity/v3/projects => generated 383 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:42:22.571384 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71399b7-2a8a-4df5-bce8-1c3d1abf480a admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:22.616339 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-026a1779-c6e0-49fa-8362-b3bd9258335d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b02a417904a474abcab1eec5cea9c7e', 'request_id': 'req-026a1779-c6e0-49fa-8362-b3bd9258335d'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:22.622424 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-026a1779-c6e0-49fa-8362-b3bd9258335d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b02a417904a474abcab1eec5cea9c7e', 'request_id': 'req-026a1779-c6e0-49fa-8362-b3bd9258335d'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:22.628795 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-026a1779-c6e0-49fa-8362-b3bd9258335d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b02a417904a474abcab1eec5cea9c7e', 'request_id': 'req-026a1779-c6e0-49fa-8362-b3bd9258335d'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:22.633384 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71399b7-2a8a-4df5-bce8-1c3d1abf480a admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:22.691039 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6df32f-4e43-4cd2-9ae8-a5bd121a6c18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:22.691711 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6df32f-4e43-4cd2-9ae8-a5bd121a6c18 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:22.691995 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6df32f-4e43-4cd2-9ae8-a5bd121a6c18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:22.692271 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6df32f-4e43-4cd2-9ae8-a5bd121a6c18 admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:22.692961 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6df32f-4e43-4cd2-9ae8-a5bd121a6c18 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:22.694928 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6df32f-4e43-4cd2-9ae8-a5bd121a6c18 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:22.703162 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-026a1779-c6e0-49fa-8362-b3bd9258335d admin admin] The token cache is being invalidate because project 4b02a417904a474abcab1eec5cea9c7e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 02:42:22.703435 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-026a1779-c6e0-49fa-8362-b3bd9258335d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:22.704291 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-026a1779-c6e0-49fa-8362-b3bd9258335d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:22.705850 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1187/2403] 10.222.0.66 () {58 vars in 1232 bytes} [Mon Jul 18 02:42:22 2022] DELETE /identity/v3/projects/4b02a417904a474abcab1eec5cea9c7e => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 02:42:22.710564 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dc8af7-20a0-466f-9fc9-c22849354355 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:22.710934 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dc8af7-20a0-466f-9fc9-c22849354355 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:22.711216 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dc8af7-20a0-466f-9fc9-c22849354355 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:22.729671 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-30dc8af7-20a0-466f-9fc9-c22849354355 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:42:22.730926 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-30dc8af7-20a0-466f-9fc9-c22849354355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:22.739012 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1188/2404] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:42:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 02:42:22.746554 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7495614-3082-4b35-8a6f-f2c889136d98 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:22.746843 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7495614-3082-4b35-8a6f-f2c889136d98 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:22.747126 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7495614-3082-4b35-8a6f-f2c889136d98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:22.764795 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-b7495614-3082-4b35-8a6f-f2c889136d98 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:42:22.765994 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-b7495614-3082-4b35-8a6f-f2c889136d98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:22.773819 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1189/2405] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:42:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 02:42:22.778805 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7495614-3082-4b35-8a6f-f2c889136d98 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:22.787694 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7495614-3082-4b35-8a6f-f2c889136d98 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:22.793986 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1217/2406] 10.222.0.66 () {60 vars in 1143 bytes} [Mon Jul 18 02:42:22 2022] POST /identity/v3/users => generated 298 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 02:42:22.794155 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682eaf55-caf4-4e89-861f-383d5e129e0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:22.794155 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682eaf55-caf4-4e89-861f-383d5e129e0d admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:22.794155 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682eaf55-caf4-4e89-861f-383d5e129e0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:22.794155 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682eaf55-caf4-4e89-861f-383d5e129e0d admin admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:22.794515 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682eaf55-caf4-4e89-861f-383d5e129e0d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:42:22.794515 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682eaf55-caf4-4e89-861f-383d5e129e0d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:22.795017 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6df32f-4e43-4cd2-9ae8-a5bd121a6c18 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:22.796122 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682eaf55-caf4-4e89-861f-383d5e129e0d admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:22.796667 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-bc6df32f-4e43-4cd2-9ae8-a5bd121a6c18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:22.811669 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1190/2407] 10.222.0.66 () {58 vars in 1152 bytes} [Mon Jul 18 02:42:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 02:42:22.811909 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6df32f-4e43-4cd2-9ae8-a5bd121a6c18 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:22.812591 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682eaf55-caf4-4e89-861f-383d5e129e0d admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:22.814352 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a991a5-b6cf-4b06-b677-0251c74be8f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:22.815025 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a991a5-b6cf-4b06-b677-0251c74be8f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:22.815314 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a991a5-b6cf-4b06-b677-0251c74be8f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:22.815632 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a991a5-b6cf-4b06-b677-0251c74be8f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:22.816381 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a991a5-b6cf-4b06-b677-0251c74be8f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:22.818348 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a991a5-b6cf-4b06-b677-0251c74be8f1 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:22.822904 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682eaf55-caf4-4e89-861f-383d5e129e0d admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:22.828187 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dbfdc4-3f00-481a-8a9d-d55705b566b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:22.828827 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dbfdc4-3f00-481a-8a9d-d55705b566b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:22.829111 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dbfdc4-3f00-481a-8a9d-d55705b566b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:22.829392 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dbfdc4-3f00-481a-8a9d-d55705b566b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:22.830100 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63dbfdc4-3f00-481a-8a9d-d55705b566b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:22.833707 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1218/2408] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:42:22 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 02:42:22.833869 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63dbfdc4-3f00-481a-8a9d-d55705b566b4 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:22.834678 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a991a5-b6cf-4b06-b677-0251c74be8f1 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:22.846957 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a991a5-b6cf-4b06-b677-0251c74be8f1 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:22.851820 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31df0c12-322f-4c47-8c41-4aa7f0b7d1e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:22.852719 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31df0c12-322f-4c47-8c41-4aa7f0b7d1e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:22.853002 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31df0c12-322f-4c47-8c41-4aa7f0b7d1e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:22.853278 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31df0c12-322f-4c47-8c41-4aa7f0b7d1e7 admin admin] PATH_INFO: `/v3/projects/97bc08c811874818b4a9d696b98cddf1/users/3c84f960831748d499d65b3b7df192c1/roles/d4aa3324f0c84887b41d0dc4acf7f30e` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:22.868011 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31df0c12-322f-4c47-8c41-4aa7f0b7d1e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=97bc08c811874818b4a9d696b98cddf1, user_id=3c84f960831748d499d65b3b7df192c1, role_id=d4aa3324f0c84887b41d0dc4acf7f30e)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:22.870754 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31df0c12-322f-4c47-8c41-4aa7f0b7d1e7 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:22.959701 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1191/2409] 10.222.0.66 () {60 vars in 1152 bytes} [Mon Jul 18 02:42:22 2022] POST /identity/v3/projects => generated 365 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 02:42:22.964126 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dbfdc4-3f00-481a-8a9d-d55705b566b4 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:22.967038 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:42:22.972707 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1219/2410] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:42:22 2022] PUT /identity/v3/projects/97bc08c811874818b4a9d696b98cddf1/users/3c84f960831748d499d65b3b7df192c1/roles/d4aa3324f0c84887b41d0dc4acf7f30e => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 02:42:22.976402 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31df0c12-322f-4c47-8c41-4aa7f0b7d1e7 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:23.026065 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dbfdc4-3f00-481a-8a9d-d55705b566b4 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:23.038253 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31df0c12-322f-4c47-8c41-4aa7f0b7d1e7 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:23.043172 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1884c9ca-ee3f-4d6e-8d7c-340443911a81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:23.043848 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1884c9ca-ee3f-4d6e-8d7c-340443911a81 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:23.044108 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1884c9ca-ee3f-4d6e-8d7c-340443911a81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:23.044379 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1884c9ca-ee3f-4d6e-8d7c-340443911a81 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:23.045015 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:42:23.045246 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1884c9ca-ee3f-4d6e-8d7c-340443911a81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:23.046797 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1884c9ca-ee3f-4d6e-8d7c-340443911a81 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:23.056636 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1220/2411] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:42:22 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 02:42:23.061464 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1884c9ca-ee3f-4d6e-8d7c-340443911a81 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:23.071338 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1884c9ca-ee3f-4d6e-8d7c-340443911a81 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:23.076780 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1053239d-be4f-4c0c-babe-b4a9d1d7346a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:23.077677 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1053239d-be4f-4c0c-babe-b4a9d1d7346a admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:23.077966 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1053239d-be4f-4c0c-babe-b4a9d1d7346a admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:23.078273 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1053239d-be4f-4c0c-babe-b4a9d1d7346a admin admin] PATH_INFO: `/v3/projects/97bc08c811874818b4a9d696b98cddf1/users/3c84f960831748d499d65b3b7df192c1/roles/d4aa3324f0c84887b41d0dc4acf7f30e` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:23.080360 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1053239d-be4f-4c0c-babe-b4a9d1d7346a admin admin] RBAC: Authorizing `identity:create_grant(project_id=97bc08c811874818b4a9d696b98cddf1, user_id=3c84f960831748d499d65b3b7df192c1, role_id=d4aa3324f0c84887b41d0dc4acf7f30e)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:23.083088 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1053239d-be4f-4c0c-babe-b4a9d1d7346a admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:23.085927 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984e3c3d-7082-4070-8a32-ac1818f85096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:23.086633 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984e3c3d-7082-4070-8a32-ac1818f85096 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:23.090354 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984e3c3d-7082-4070-8a32-ac1818f85096 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:23.090354 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984e3c3d-7082-4070-8a32-ac1818f85096 admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:23.090354 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984e3c3d-7082-4070-8a32-ac1818f85096 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:23.090354 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984e3c3d-7082-4070-8a32-ac1818f85096 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:23.093834 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1221/2412] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:42:23 2022] PUT /identity/v3/projects/97bc08c811874818b4a9d696b98cddf1/users/3c84f960831748d499d65b3b7df192c1/roles/d4aa3324f0c84887b41d0dc4acf7f30e => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 02:42:23.104445 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d70e244-645f-411e-b170-10d9d5127236 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:23.104726 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d70e244-645f-411e-b170-10d9d5127236 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:23.105020 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d70e244-645f-411e-b170-10d9d5127236 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:23.133020 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-7d70e244-645f-411e-b170-10d9d5127236 None None] MFA Rules not processed for user `3c84f960831748d499d65b3b7df192c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:42:23.134270 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-7d70e244-645f-411e-b170-10d9d5127236 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:23.200408 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1222/2413] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:42:23 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:42:23.215804 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d70e244-645f-411e-b170-10d9d5127236 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:23.215804 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-7d70e244-645f-411e-b170-10d9d5127236 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:23.220751 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1192/2414] 10.222.0.66 () {60 vars in 1143 bytes} [Mon Jul 18 02:42:22 2022] POST /identity/v3/users => generated 283 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:42:23.221944 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984e3c3d-7082-4070-8a32-ac1818f85096 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:23.231698 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984e3c3d-7082-4070-8a32-ac1818f85096 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:23.271190 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d70e244-645f-411e-b170-10d9d5127236 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:23.274367 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2953505b-d526-495c-836f-130a1ac93653 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd49388facdf446885e55b45ef91f01e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:23.275043 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2953505b-d526-495c-836f-130a1ac93653 service cinder] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:23.275309 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2953505b-d526-495c-836f-130a1ac93653 service cinder] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:23.275610 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2953505b-d526-495c-836f-130a1ac93653 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:23.287450 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2953505b-d526-495c-836f-130a1ac93653 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:23.288901 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2953505b-d526-495c-836f-130a1ac93653 service cinder] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:23.306740 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b276de95-13b5-4a95-86be-d0b97987088a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:23.306740 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b276de95-13b5-4a95-86be-d0b97987088a admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:23.306740 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b276de95-13b5-4a95-86be-d0b97987088a admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:23.306740 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b276de95-13b5-4a95-86be-d0b97987088a admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:23.307068 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b276de95-13b5-4a95-86be-d0b97987088a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:23.307068 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b276de95-13b5-4a95-86be-d0b97987088a admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:23.307222 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1223/2415] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:42:23 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 02:42:23.309881 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1193/2416] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:42:23 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:42:23.314217 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2953505b-d526-495c-836f-130a1ac93653 service cinder] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:23.326288 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2953505b-d526-495c-836f-130a1ac93653 service cinder] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:23.331103 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:42:23.336353 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5f3c73-f8f4-43e4-b05a-deb2f18b4b03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:23.336353 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5f3c73-f8f4-43e4-b05a-deb2f18b4b03 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:23.336731 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5f3c73-f8f4-43e4-b05a-deb2f18b4b03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:23.337174 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5f3c73-f8f4-43e4-b05a-deb2f18b4b03 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:23.338063 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5f3c73-f8f4-43e4-b05a-deb2f18b4b03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:23.340753 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5f3c73-f8f4-43e4-b05a-deb2f18b4b03 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:23.352383 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1224/2417] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:42:23 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:42:23.356736 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b276de95-13b5-4a95-86be-d0b97987088a admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:23.370293 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b276de95-13b5-4a95-86be-d0b97987088a admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:23.381670 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3470d830-3ea1-4463-9a7f-9811854e28f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:23.383093 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3470d830-3ea1-4463-9a7f-9811854e28f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:23.383468 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3470d830-3ea1-4463-9a7f-9811854e28f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:23.383824 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3470d830-3ea1-4463-9a7f-9811854e28f5 admin admin] PATH_INFO: `/v3/projects/f9cad813a9304bd897bd93dc2277368b/users/1c9f18acfaab426abc52542ff5ddbee9/roles/65feabf3b27944a8b01a86c8ac608457` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:23.408120 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3470d830-3ea1-4463-9a7f-9811854e28f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9cad813a9304bd897bd93dc2277368b, user_id=1c9f18acfaab426abc52542ff5ddbee9, role_id=65feabf3b27944a8b01a86c8ac608457)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:23.411092 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3470d830-3ea1-4463-9a7f-9811854e28f5 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:23.413471 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5f3c73-f8f4-43e4-b05a-deb2f18b4b03 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:23.415560 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-6d5f3c73-f8f4-43e4-b05a-deb2f18b4b03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:23.421238 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:42:23.428791 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5f3c73-f8f4-43e4-b05a-deb2f18b4b03 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:23.432444 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd963c8f-0fc8-4ddc-9421-30fa40ccd79f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd49388facdf446885e55b45ef91f01e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:23.433081 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd963c8f-0fc8-4ddc-9421-30fa40ccd79f service cinder] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:23.433983 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd963c8f-0fc8-4ddc-9421-30fa40ccd79f service cinder] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:23.434286 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd963c8f-0fc8-4ddc-9421-30fa40ccd79f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:23.436073 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-bd963c8f-0fc8-4ddc-9421-30fa40ccd79f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:23.493699 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd963c8f-0fc8-4ddc-9421-30fa40ccd79f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:23.495102 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd963c8f-0fc8-4ddc-9421-30fa40ccd79f service cinder] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:23.504634 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1225/2418] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:42:23 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 02:42:23.520462 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1194/2419] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:42:23 2022] PUT /identity/v3/projects/f9cad813a9304bd897bd93dc2277368b/users/1c9f18acfaab426abc52542ff5ddbee9/roles/65feabf3b27944a8b01a86c8ac608457 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:42:23.531411 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf33fa5-fe0d-4680-a701-d3d327b00aeb None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:23.531752 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf33fa5-fe0d-4680-a701-d3d327b00aeb None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:23.532035 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf33fa5-fe0d-4680-a701-d3d327b00aeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:23.560464 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-bcf33fa5-fe0d-4680-a701-d3d327b00aeb None None] MFA Rules not processed for user `1c9f18acfaab426abc52542ff5ddbee9`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:42:23.561957 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-bcf33fa5-fe0d-4680-a701-d3d327b00aeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:23.617500 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:42:23.628966 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1226/2420] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:42:23 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:42:23.634443 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3470d830-3ea1-4463-9a7f-9811854e28f5 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:23.703159 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3470d830-3ea1-4463-9a7f-9811854e28f5 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:23.707950 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914aac8e-7252-4f7f-9416-0bbfbc7be03e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:23.708612 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914aac8e-7252-4f7f-9416-0bbfbc7be03e admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:23.708896 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914aac8e-7252-4f7f-9416-0bbfbc7be03e admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:23.709174 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914aac8e-7252-4f7f-9416-0bbfbc7be03e admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:23.709875 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914aac8e-7252-4f7f-9416-0bbfbc7be03e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:23.711873 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914aac8e-7252-4f7f-9416-0bbfbc7be03e admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:23.826401 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1195/2421] 10.222.0.66 () {60 vars in 1152 bytes} [Mon Jul 18 02:42:23 2022] POST /identity/v3/projects => generated 363 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:42:23.830308 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf33fa5-fe0d-4680-a701-d3d327b00aeb None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:23.841808 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf33fa5-fe0d-4680-a701-d3d327b00aeb None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:23.897176 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b92b42-3e33-4570-ba43-51b95a9239b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:23.897842 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b92b42-3e33-4570-ba43-51b95a9239b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:23.898150 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b92b42-3e33-4570-ba43-51b95a9239b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:23.898451 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b92b42-3e33-4570-ba43-51b95a9239b2 admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:23.899117 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b92b42-3e33-4570-ba43-51b95a9239b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:23.901037 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b92b42-3e33-4570-ba43-51b95a9239b2 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:24.017203 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1227/2422] 10.222.0.66 () {60 vars in 1143 bytes} [Mon Jul 18 02:42:23 2022] POST /identity/v3/users => generated 288 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 02:42:24.021308 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914aac8e-7252-4f7f-9416-0bbfbc7be03e admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:24.032540 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914aac8e-7252-4f7f-9416-0bbfbc7be03e admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:24.037159 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fa73aa-8d53-4872-9eb5-543f1832e66f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:24.037813 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fa73aa-8d53-4872-9eb5-543f1832e66f admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:24.038201 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fa73aa-8d53-4872-9eb5-543f1832e66f admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:24.038501 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fa73aa-8d53-4872-9eb5-543f1832e66f admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:24.039230 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fa73aa-8d53-4872-9eb5-543f1832e66f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:24.040855 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fa73aa-8d53-4872-9eb5-543f1832e66f admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:24.052595 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1196/2423] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:42:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:42:24.056966 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b92b42-3e33-4570-ba43-51b95a9239b2 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:24.067915 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b92b42-3e33-4570-ba43-51b95a9239b2 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:24.072672 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f089588b-a5d9-4adc-9397-27bc25ba84d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:24.073603 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f089588b-a5d9-4adc-9397-27bc25ba84d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:24.074515 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f089588b-a5d9-4adc-9397-27bc25ba84d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:24.074803 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f089588b-a5d9-4adc-9397-27bc25ba84d1 admin admin] PATH_INFO: `/v3/projects/87d1ef70a8f94526a0eb1d3f4bc61c56/users/123b275490644613a2228f3856aa8f05/roles/d4aa3324f0c84887b41d0dc4acf7f30e` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:24.090221 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f089588b-a5d9-4adc-9397-27bc25ba84d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=87d1ef70a8f94526a0eb1d3f4bc61c56, user_id=123b275490644613a2228f3856aa8f05, role_id=d4aa3324f0c84887b41d0dc4acf7f30e)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:24.092853 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f089588b-a5d9-4adc-9397-27bc25ba84d1 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:24.169964 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1228/2424] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:42:24 2022] PUT /identity/v3/projects/87d1ef70a8f94526a0eb1d3f4bc61c56/users/123b275490644613a2228f3856aa8f05/roles/d4aa3324f0c84887b41d0dc4acf7f30e => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 02:42:24.174025 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fa73aa-8d53-4872-9eb5-543f1832e66f admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:24.184195 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fa73aa-8d53-4872-9eb5-543f1832e66f admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:24.242076 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5170b045-9213-4327-827e-e41ad9568ce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:24.242766 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5170b045-9213-4327-827e-e41ad9568ce3 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:24.243047 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5170b045-9213-4327-827e-e41ad9568ce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:24.243319 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5170b045-9213-4327-827e-e41ad9568ce3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:24.244033 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5170b045-9213-4327-827e-e41ad9568ce3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:24.245732 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5170b045-9213-4327-827e-e41ad9568ce3 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:24.260277 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1197/2425] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:42:24 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 02:42:24.261945 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f089588b-a5d9-4adc-9397-27bc25ba84d1 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:24.279407 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f089588b-a5d9-4adc-9397-27bc25ba84d1 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:24.284257 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378cfca7-069b-4262-8f9f-31142f947756 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:24.285138 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378cfca7-069b-4262-8f9f-31142f947756 admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:24.285445 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378cfca7-069b-4262-8f9f-31142f947756 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:24.285718 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378cfca7-069b-4262-8f9f-31142f947756 admin admin] PATH_INFO: `/v3/projects/87d1ef70a8f94526a0eb1d3f4bc61c56/users/123b275490644613a2228f3856aa8f05/roles/d4aa3324f0c84887b41d0dc4acf7f30e` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:24.287714 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378cfca7-069b-4262-8f9f-31142f947756 admin admin] RBAC: Authorizing `identity:create_grant(project_id=87d1ef70a8f94526a0eb1d3f4bc61c56, user_id=123b275490644613a2228f3856aa8f05, role_id=d4aa3324f0c84887b41d0dc4acf7f30e)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:24.291337 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378cfca7-069b-4262-8f9f-31142f947756 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:24.303249 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1229/2426] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:42:24 2022] PUT /identity/v3/projects/87d1ef70a8f94526a0eb1d3f4bc61c56/users/123b275490644613a2228f3856aa8f05/roles/d4aa3324f0c84887b41d0dc4acf7f30e => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:42:24.314721 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d48ff7b-4613-4a73-b802-513ab172cc32 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:24.314721 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d48ff7b-4613-4a73-b802-513ab172cc32 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:24.314721 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d48ff7b-4613-4a73-b802-513ab172cc32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:24.341391 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-0d48ff7b-4613-4a73-b802-513ab172cc32 None None] MFA Rules not processed for user `123b275490644613a2228f3856aa8f05`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:42:24.342609 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-0d48ff7b-4613-4a73-b802-513ab172cc32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:24.377177 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:42:24.416796 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1198/2427] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:42:24 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:42:24.428644 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378cfca7-069b-4262-8f9f-31142f947756 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:24.485331 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378cfca7-069b-4262-8f9f-31142f947756 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:24.488957 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf0875a-4ea3-4ef8-a478-cc9d18bf78bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd49388facdf446885e55b45ef91f01e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:24.489598 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf0875a-4ea3-4ef8-a478-cc9d18bf78bf service cinder] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:24.489880 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf0875a-4ea3-4ef8-a478-cc9d18bf78bf service cinder] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:24.490200 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf0875a-4ea3-4ef8-a478-cc9d18bf78bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:24.499768 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf0875a-4ea3-4ef8-a478-cc9d18bf78bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:24.501215 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf0875a-4ea3-4ef8-a478-cc9d18bf78bf service cinder] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:24.510496 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1230/2428] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:42:24 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:42:24.716114 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d48ff7b-4613-4a73-b802-513ab172cc32 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:24.725685 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d48ff7b-4613-4a73-b802-513ab172cc32 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:24.729318 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e356bd8b-baf8-4ff2-af3d-31fc1a235a2b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd49388facdf446885e55b45ef91f01e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:24.729955 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e356bd8b-baf8-4ff2-af3d-31fc1a235a2b service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:24.730403 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e356bd8b-baf8-4ff2-af3d-31fc1a235a2b service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:24.730683 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e356bd8b-baf8-4ff2-af3d-31fc1a235a2b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:24.796007 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e356bd8b-baf8-4ff2-af3d-31fc1a235a2b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:24.798930 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e356bd8b-baf8-4ff2-af3d-31fc1a235a2b service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:24.805543 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf0875a-4ea3-4ef8-a478-cc9d18bf78bf service cinder] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:24.806317 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1199/2429] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:42:24 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:42:24.806962 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-adf0875a-4ea3-4ef8-a478-cc9d18bf78bf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:24.860745 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:42:24.864562 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf0875a-4ea3-4ef8-a478-cc9d18bf78bf service cinder] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:24.868156 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3999ebb-b086-4b6c-9da8-3035bc4857ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eaf19a259c2741cea375a38763e33cdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:24.868803 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3999ebb-b086-4b6c-9da8-3035bc4857ec service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:24.869090 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3999ebb-b086-4b6c-9da8-3035bc4857ec service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:24.869369 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3999ebb-b086-4b6c-9da8-3035bc4857ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:24.871814 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-b3999ebb-b086-4b6c-9da8-3035bc4857ec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:24.933263 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3999ebb-b086-4b6c-9da8-3035bc4857ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:24.935257 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3999ebb-b086-4b6c-9da8-3035bc4857ec service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:24.945206 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1231/2430] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:42:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 02:42:26.773779 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e356bd8b-baf8-4ff2-af3d-31fc1a235a2b service cinder] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:26.791080 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e356bd8b-baf8-4ff2-af3d-31fc1a235a2b service cinder] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:26.799235 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ec0330-ffb8-4409-8e20-a4e507e97089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:26.800929 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ec0330-ffb8-4409-8e20-a4e507e97089 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:26.801496 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ec0330-ffb8-4409-8e20-a4e507e97089 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:26.801803 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ec0330-ffb8-4409-8e20-a4e507e97089 admin admin] PATH_INFO: `/v3/users/22e282b85acc4dd18c0545a42fa10d36` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:26.803834 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ec0330-ffb8-4409-8e20-a4e507e97089 admin admin] RBAC: Authorizing `identity:delete_user(user_id=22e282b85acc4dd18c0545a42fa10d36)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:26.806396 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ec0330-ffb8-4409-8e20-a4e507e97089 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:27.088607 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-79ec0330-ffb8-4409-8e20-a4e507e97089 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22e282b85acc4dd18c0545a42fa10d36'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:27.177114 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-79ec0330-ffb8-4409-8e20-a4e507e97089 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22e282b85acc4dd18c0545a42fa10d36'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:27.177824 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-79ec0330-ffb8-4409-8e20-a4e507e97089 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22e282b85acc4dd18c0545a42fa10d36'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:27.196029 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-79ec0330-ffb8-4409-8e20-a4e507e97089 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22e282b85acc4dd18c0545a42fa10d36'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:27.201461 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-79ec0330-ffb8-4409-8e20-a4e507e97089 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22e282b85acc4dd18c0545a42fa10d36'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:27.213934 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1200/2431] 10.222.0.66 () {58 vars in 1223 bytes} [Mon Jul 18 02:42:26 2022] DELETE /identity/v3/users/22e282b85acc4dd18c0545a42fa10d36 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:42:27.219622 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f6ba8a-662a-4837-821d-701ab5639651 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:27.220104 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f6ba8a-662a-4837-821d-701ab5639651 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:27.220216 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f6ba8a-662a-4837-821d-701ab5639651 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:27.239675 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-16f6ba8a-662a-4837-821d-701ab5639651 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:42:27.240929 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-16f6ba8a-662a-4837-821d-701ab5639651 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:27.315741 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1232/2432] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:42:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:42:27.324739 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ec0330-ffb8-4409-8e20-a4e507e97089 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:27.326210 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-79ec0330-ffb8-4409-8e20-a4e507e97089 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:27.507888 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:42:27.522133 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ec0330-ffb8-4409-8e20-a4e507e97089 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:27.525796 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffe642d-454b-4a9b-a696-1696cc87a928 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eaf19a259c2741cea375a38763e33cdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:27.526523 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffe642d-454b-4a9b-a696-1696cc87a928 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:27.526807 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffe642d-454b-4a9b-a696-1696cc87a928 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:27.527701 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffe642d-454b-4a9b-a696-1696cc87a928 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:27.538235 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffe642d-454b-4a9b-a696-1696cc87a928 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:27.538896 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffe642d-454b-4a9b-a696-1696cc87a928 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:27.549655 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1201/2433] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:42:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 02:42:28.202907 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f6ba8a-662a-4837-821d-701ab5639651 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:28.204436 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-16f6ba8a-662a-4837-821d-701ab5639651 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:28.218007 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f6ba8a-662a-4837-821d-701ab5639651 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:28.225382 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f273e4f-7487-4903-870b-1cb1f9888265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:28.226295 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f273e4f-7487-4903-870b-1cb1f9888265 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:28.226659 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f273e4f-7487-4903-870b-1cb1f9888265 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:28.226949 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f273e4f-7487-4903-870b-1cb1f9888265 admin admin] PATH_INFO: `/v3/projects/56b9f450bc2c4910b141e7412653df30` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:28.228147 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f273e4f-7487-4903-870b-1cb1f9888265 admin admin] RBAC: Authorizing `identity:delete_project(project_id=56b9f450bc2c4910b141e7412653df30)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:28.230223 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f273e4f-7487-4903-870b-1cb1f9888265 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:28.426669 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-9f273e4f-7487-4903-870b-1cb1f9888265 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56b9f450bc2c4910b141e7412653df30', 'request_id': 'req-9f273e4f-7487-4903-870b-1cb1f9888265'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:28.434245 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-9f273e4f-7487-4903-870b-1cb1f9888265 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56b9f450bc2c4910b141e7412653df30', 'request_id': 'req-9f273e4f-7487-4903-870b-1cb1f9888265'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:28.559481 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-9f273e4f-7487-4903-870b-1cb1f9888265 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56b9f450bc2c4910b141e7412653df30', 'request_id': 'req-9f273e4f-7487-4903-870b-1cb1f9888265'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:28.572441 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-9f273e4f-7487-4903-870b-1cb1f9888265 admin admin] The token cache is being invalidate because project 56b9f450bc2c4910b141e7412653df30 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 02:42:28.572734 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-9f273e4f-7487-4903-870b-1cb1f9888265 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:28.573389 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-9f273e4f-7487-4903-870b-1cb1f9888265 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:28.575497 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1233/2434] 10.222.0.66 () {58 vars in 1232 bytes} [Mon Jul 18 02:42:28 2022] DELETE /identity/v3/projects/56b9f450bc2c4910b141e7412653df30 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:42:28.579554 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffe642d-454b-4a9b-a696-1696cc87a928 service neutron] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:28.593110 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffe642d-454b-4a9b-a696-1696cc87a928 service neutron] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:28.653161 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea263fe-7604-458f-a4ff-a70be94db835 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:28.654251 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea263fe-7604-458f-a4ff-a70be94db835 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:28.654597 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea263fe-7604-458f-a4ff-a70be94db835 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:28.655006 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea263fe-7604-458f-a4ff-a70be94db835 admin admin] PATH_INFO: `/v3/users/3c84f960831748d499d65b3b7df192c1` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:28.656367 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea263fe-7604-458f-a4ff-a70be94db835 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c84f960831748d499d65b3b7df192c1)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:28.658335 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea263fe-7604-458f-a4ff-a70be94db835 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:28.665927 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f273e4f-7487-4903-870b-1cb1f9888265 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:28.671215 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-9f273e4f-7487-4903-870b-1cb1f9888265 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:28.683932 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f273e4f-7487-4903-870b-1cb1f9888265 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:28.689485 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d06ad3-4d94-443b-a110-d4bb02cc6c1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:28.690394 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d06ad3-4d94-443b-a110-d4bb02cc6c1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:28.690678 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d06ad3-4d94-443b-a110-d4bb02cc6c1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:28.694217 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d06ad3-4d94-443b-a110-d4bb02cc6c1e admin admin] PATH_INFO: `/v3/users/879f7a4cf6664e76bddffc804658b250` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:28.694217 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d06ad3-4d94-443b-a110-d4bb02cc6c1e admin admin] RBAC: Authorizing `identity:delete_user(user_id=879f7a4cf6664e76bddffc804658b250)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:28.694217 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d06ad3-4d94-443b-a110-d4bb02cc6c1e admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:28.717080 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:42:28.898847 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-5ea263fe-7604-458f-a4ff-a70be94db835 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c84f960831748d499d65b3b7df192c1'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:28.910464 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-11d06ad3-4d94-443b-a110-d4bb02cc6c1e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '879f7a4cf6664e76bddffc804658b250'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:28.975648 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-5ea263fe-7604-458f-a4ff-a70be94db835 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c84f960831748d499d65b3b7df192c1'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:28.976179 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-5ea263fe-7604-458f-a4ff-a70be94db835 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c84f960831748d499d65b3b7df192c1'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:28.989960 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-5ea263fe-7604-458f-a4ff-a70be94db835 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c84f960831748d499d65b3b7df192c1'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:28.990758 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-5ea263fe-7604-458f-a4ff-a70be94db835 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c84f960831748d499d65b3b7df192c1'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:29.004747 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1202/2435] 10.222.0.66 () {58 vars in 1223 bytes} [Mon Jul 18 02:42:28 2022] DELETE /identity/v3/users/3c84f960831748d499d65b3b7df192c1 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:42:29.050006 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-11d06ad3-4d94-443b-a110-d4bb02cc6c1e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '879f7a4cf6664e76bddffc804658b250'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:29.061967 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-11d06ad3-4d94-443b-a110-d4bb02cc6c1e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '879f7a4cf6664e76bddffc804658b250'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:29.065265 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-11d06ad3-4d94-443b-a110-d4bb02cc6c1e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '879f7a4cf6664e76bddffc804658b250'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:29.084701 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-11d06ad3-4d94-443b-a110-d4bb02cc6c1e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '879f7a4cf6664e76bddffc804658b250'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:29.093239 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1234/2436] 10.222.0.66 () {58 vars in 1223 bytes} [Mon Jul 18 02:42:28 2022] DELETE /identity/v3/users/879f7a4cf6664e76bddffc804658b250 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 02:42:29.373815 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea263fe-7604-458f-a4ff-a70be94db835 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:29.376941 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-5ea263fe-7604-458f-a4ff-a70be94db835 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:29.443881 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea263fe-7604-458f-a4ff-a70be94db835 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:29.448725 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:42:29.492950 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d06ad3-4d94-443b-a110-d4bb02cc6c1e admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:29.495233 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-11d06ad3-4d94-443b-a110-d4bb02cc6c1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:29.545412 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c358166-f9c8-4563-9a58-da5cfc7224ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:29.547414 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c358166-f9c8-4563-9a58-da5cfc7224ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:29.550536 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c358166-f9c8-4563-9a58-da5cfc7224ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:29.550829 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c358166-f9c8-4563-9a58-da5cfc7224ab admin admin] PATH_INFO: `/v3/projects/85204e1ae6844b70b45f8452887d5385` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:29.552538 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c358166-f9c8-4563-9a58-da5cfc7224ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=85204e1ae6844b70b45f8452887d5385)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:29.555693 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c358166-f9c8-4563-9a58-da5cfc7224ab admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:29.602628 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d06ad3-4d94-443b-a110-d4bb02cc6c1e admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:29.608520 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4febc9-573a-4732-82ff-e4cfb69ee241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:29.610112 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4febc9-573a-4732-82ff-e4cfb69ee241 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:29.614283 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4febc9-573a-4732-82ff-e4cfb69ee241 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:29.614610 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4febc9-573a-4732-82ff-e4cfb69ee241 admin admin] PATH_INFO: `/v3/projects/97bc08c811874818b4a9d696b98cddf1` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:29.615921 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4febc9-573a-4732-82ff-e4cfb69ee241 admin admin] RBAC: Authorizing `identity:delete_project(project_id=97bc08c811874818b4a9d696b98cddf1)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:29.618518 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4febc9-573a-4732-82ff-e4cfb69ee241 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:29.702587 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-9c358166-f9c8-4563-9a58-da5cfc7224ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85204e1ae6844b70b45f8452887d5385', 'request_id': 'req-9c358166-f9c8-4563-9a58-da5cfc7224ab'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:29.708213 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-9c358166-f9c8-4563-9a58-da5cfc7224ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85204e1ae6844b70b45f8452887d5385', 'request_id': 'req-9c358166-f9c8-4563-9a58-da5cfc7224ab'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:29.717383 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-9c358166-f9c8-4563-9a58-da5cfc7224ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85204e1ae6844b70b45f8452887d5385', 'request_id': 'req-9c358166-f9c8-4563-9a58-da5cfc7224ab'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:29.772129 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-df4febc9-573a-4732-82ff-e4cfb69ee241 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97bc08c811874818b4a9d696b98cddf1', 'request_id': 'req-df4febc9-573a-4732-82ff-e4cfb69ee241'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:29.778965 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-df4febc9-573a-4732-82ff-e4cfb69ee241 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97bc08c811874818b4a9d696b98cddf1', 'request_id': 'req-df4febc9-573a-4732-82ff-e4cfb69ee241'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:29.857483 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-9c358166-f9c8-4563-9a58-da5cfc7224ab admin admin] The token cache is being invalidate because project 85204e1ae6844b70b45f8452887d5385 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 02:42:29.857871 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-9c358166-f9c8-4563-9a58-da5cfc7224ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:29.858533 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-9c358166-f9c8-4563-9a58-da5cfc7224ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:29.860084 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1203/2437] 10.222.0.66 () {58 vars in 1232 bytes} [Mon Jul 18 02:42:29 2022] DELETE /identity/v3/projects/85204e1ae6844b70b45f8452887d5385 => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:42:29.865914 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecfb580-4181-4dbd-8ae0-e2a176a899a2 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:29.866251 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecfb580-4181-4dbd-8ae0-e2a176a899a2 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:29.866561 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecfb580-4181-4dbd-8ae0-e2a176a899a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:29.884092 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-5ecfb580-4181-4dbd-8ae0-e2a176a899a2 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:42:29.885289 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-5ecfb580-4181-4dbd-8ae0-e2a176a899a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:29.908301 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-df4febc9-573a-4732-82ff-e4cfb69ee241 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97bc08c811874818b4a9d696b98cddf1', 'request_id': 'req-df4febc9-573a-4732-82ff-e4cfb69ee241'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:29.920923 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-df4febc9-573a-4732-82ff-e4cfb69ee241 admin admin] The token cache is being invalidate because project 97bc08c811874818b4a9d696b98cddf1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 02:42:29.921342 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-df4febc9-573a-4732-82ff-e4cfb69ee241 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:29.921994 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-df4febc9-573a-4732-82ff-e4cfb69ee241 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:29.923596 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1235/2438] 10.222.0.66 () {58 vars in 1232 bytes} [Mon Jul 18 02:42:29 2022] DELETE /identity/v3/projects/97bc08c811874818b4a9d696b98cddf1 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:42:29.928305 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34498283-5a07-45d0-9936-07e101f347ac None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:29.928673 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34498283-5a07-45d0-9936-07e101f347ac None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:29.928957 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34498283-5a07-45d0-9936-07e101f347ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:29.945434 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1204/2439] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:42:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 02:42:29.947538 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-34498283-5a07-45d0-9936-07e101f347ac None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:42:29.948589 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-34498283-5a07-45d0-9936-07e101f347ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:29.953791 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f693a907-393a-45d1-ad19-bd4169aaaf67 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:29.954102 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f693a907-393a-45d1-ad19-bd4169aaaf67 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:29.954390 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f693a907-393a-45d1-ad19-bd4169aaaf67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:29.956984 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1236/2440] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:42:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:42:29.965240 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0d0171-d129-4efb-8d28-9e32f92323e0 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:29.965674 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0d0171-d129-4efb-8d28-9e32f92323e0 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:29.966267 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0d0171-d129-4efb-8d28-9e32f92323e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:29.975655 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-f693a907-393a-45d1-ad19-bd4169aaaf67 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:42:29.976863 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-f693a907-393a-45d1-ad19-bd4169aaaf67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:29.984550 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-bc0d0171-d129-4efb-8d28-9e32f92323e0 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:42:29.985611 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1205/2441] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:42:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 02:42:29.985738 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-bc0d0171-d129-4efb-8d28-9e32f92323e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:29.990007 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f693a907-393a-45d1-ad19-bd4169aaaf67 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:29.993878 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1237/2442] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:42:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 02:42:29.998191 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0d0171-d129-4efb-8d28-9e32f92323e0 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:29.999016 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f693a907-393a-45d1-ad19-bd4169aaaf67 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:30.003887 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c8c1b2-4d48-45ec-b8b8-b345a0b33c17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:30.004472 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c8c1b2-4d48-45ec-b8b8-b345a0b33c17 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:30.004752 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c8c1b2-4d48-45ec-b8b8-b345a0b33c17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:30.005027 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c8c1b2-4d48-45ec-b8b8-b345a0b33c17 admin admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:30.005768 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c8c1b2-4d48-45ec-b8b8-b345a0b33c17 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:42:30.006051 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c8c1b2-4d48-45ec-b8b8-b345a0b33c17 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:30.007306 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0d0171-d129-4efb-8d28-9e32f92323e0 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:30.007680 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c8c1b2-4d48-45ec-b8b8-b345a0b33c17 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:30.012292 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7670baa0-0a96-42fd-80db-3ce1784ec54e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:30.012938 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7670baa0-0a96-42fd-80db-3ce1784ec54e admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:30.013223 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7670baa0-0a96-42fd-80db-3ce1784ec54e admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:30.016653 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7670baa0-0a96-42fd-80db-3ce1784ec54e admin admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:30.016653 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7670baa0-0a96-42fd-80db-3ce1784ec54e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:42:30.016653 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7670baa0-0a96-42fd-80db-3ce1784ec54e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:30.016653 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7670baa0-0a96-42fd-80db-3ce1784ec54e admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:30.019622 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1206/2443] 10.222.0.66 () {58 vars in 1152 bytes} [Mon Jul 18 02:42:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 02:42:30.023846 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c8c1b2-4d48-45ec-b8b8-b345a0b33c17 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:30.025852 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:42:30.034125 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1238/2444] 10.222.0.66 () {58 vars in 1152 bytes} [Mon Jul 18 02:42:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 02:42:30.036681 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c8c1b2-4d48-45ec-b8b8-b345a0b33c17 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:30.036833 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7670baa0-0a96-42fd-80db-3ce1784ec54e admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:30.041342 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ce1455-0b44-4328-938c-c67e7016c4d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:30.041997 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ce1455-0b44-4328-938c-c67e7016c4d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:30.042348 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ce1455-0b44-4328-938c-c67e7016c4d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:30.042681 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ce1455-0b44-4328-938c-c67e7016c4d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:30.043493 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ce1455-0b44-4328-938c-c67e7016c4d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:30.045489 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ce1455-0b44-4328-938c-c67e7016c4d0 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:30.046683 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7670baa0-0a96-42fd-80db-3ce1784ec54e admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:30.051280 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60679bf-0002-4f01-916b-b7a4cd833ce7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:30.051917 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60679bf-0002-4f01-916b-b7a4cd833ce7 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:30.052204 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60679bf-0002-4f01-916b-b7a4cd833ce7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:30.052485 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60679bf-0002-4f01-916b-b7a4cd833ce7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:30.053173 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60679bf-0002-4f01-916b-b7a4cd833ce7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:30.055113 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60679bf-0002-4f01-916b-b7a4cd833ce7 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:30.159780 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1207/2445] 10.222.0.66 () {60 vars in 1152 bytes} [Mon Jul 18 02:42:30 2022] POST /identity/v3/projects => generated 379 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 02:42:30.163938 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ce1455-0b44-4328-938c-c67e7016c4d0 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:30.185268 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:42:30.193718 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1239/2446] 10.222.0.66 () {60 vars in 1152 bytes} [Mon Jul 18 02:42:30 2022] POST /identity/v3/projects => generated 373 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 02:42:30.194742 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60679bf-0002-4f01-916b-b7a4cd833ce7 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:30.228089 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ce1455-0b44-4328-938c-c67e7016c4d0 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:30.255796 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60679bf-0002-4f01-916b-b7a4cd833ce7 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:30.261819 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6d01ea-86d1-4417-a425-5401f2b4be2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:30.262187 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6d01ea-86d1-4417-a425-5401f2b4be2c admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:30.263134 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6d01ea-86d1-4417-a425-5401f2b4be2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:30.263134 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6d01ea-86d1-4417-a425-5401f2b4be2c admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:30.263405 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6d01ea-86d1-4417-a425-5401f2b4be2c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:30.265327 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6d01ea-86d1-4417-a425-5401f2b4be2c admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:30.289162 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b94bdeb-a9ce-4855-b282-204876c97705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:30.289849 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b94bdeb-a9ce-4855-b282-204876c97705 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:30.290160 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b94bdeb-a9ce-4855-b282-204876c97705 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:30.290439 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b94bdeb-a9ce-4855-b282-204876c97705 admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:30.291137 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b94bdeb-a9ce-4855-b282-204876c97705 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:30.293470 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b94bdeb-a9ce-4855-b282-204876c97705 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:30.370200 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1240/2447] 10.222.0.66 () {60 vars in 1143 bytes} [Mon Jul 18 02:42:30 2022] POST /identity/v3/users => generated 287 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:42:30.378188 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1208/2448] 10.222.0.66 () {60 vars in 1143 bytes} [Mon Jul 18 02:42:30 2022] POST /identity/v3/users => generated 290 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 02:42:30.379626 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6d01ea-86d1-4417-a425-5401f2b4be2c admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:30.382160 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b94bdeb-a9ce-4855-b282-204876c97705 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:30.390981 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6d01ea-86d1-4417-a425-5401f2b4be2c admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:30.392596 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b94bdeb-a9ce-4855-b282-204876c97705 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:30.395770 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745f2850-8c8b-4a6b-b4be-1f9d3663b948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:30.399725 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745f2850-8c8b-4a6b-b4be-1f9d3663b948 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:30.399725 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745f2850-8c8b-4a6b-b4be-1f9d3663b948 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:30.399725 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745f2850-8c8b-4a6b-b4be-1f9d3663b948 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:30.399999 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c726e426-9edd-4fa8-92f5-42feff2bc619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:30.400119 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745f2850-8c8b-4a6b-b4be-1f9d3663b948 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:30.400215 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c726e426-9edd-4fa8-92f5-42feff2bc619 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:30.400215 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c726e426-9edd-4fa8-92f5-42feff2bc619 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:30.400215 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c726e426-9edd-4fa8-92f5-42feff2bc619 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:30.400463 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745f2850-8c8b-4a6b-b4be-1f9d3663b948 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:30.400566 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c726e426-9edd-4fa8-92f5-42feff2bc619 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:30.401013 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c726e426-9edd-4fa8-92f5-42feff2bc619 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:30.410288 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1241/2449] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:42:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:42:30.412426 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1209/2450] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:42:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 02:42:30.414035 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745f2850-8c8b-4a6b-b4be-1f9d3663b948 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:30.415583 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c726e426-9edd-4fa8-92f5-42feff2bc619 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:30.423888 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745f2850-8c8b-4a6b-b4be-1f9d3663b948 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:30.431476 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c726e426-9edd-4fa8-92f5-42feff2bc619 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:30.431597 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da98190a-d403-40dc-b82f-e1c597fc34ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:30.431597 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da98190a-d403-40dc-b82f-e1c597fc34ef admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:30.431597 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da98190a-d403-40dc-b82f-e1c597fc34ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:30.431597 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da98190a-d403-40dc-b82f-e1c597fc34ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:30.431986 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c112390-ca80-41bb-acb5-dbdc9bb0547e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:30.432100 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da98190a-d403-40dc-b82f-e1c597fc34ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:30.432199 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c112390-ca80-41bb-acb5-dbdc9bb0547e admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:30.432199 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c112390-ca80-41bb-acb5-dbdc9bb0547e admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:30.432199 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c112390-ca80-41bb-acb5-dbdc9bb0547e admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:30.432464 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c112390-ca80-41bb-acb5-dbdc9bb0547e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:30.434324 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da98190a-d403-40dc-b82f-e1c597fc34ef admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:30.434885 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c112390-ca80-41bb-acb5-dbdc9bb0547e admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:30.445357 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1242/2451] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:42:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 02:42:30.448795 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1210/2452] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:42:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:42:30.450247 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da98190a-d403-40dc-b82f-e1c597fc34ef admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:30.450519 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c112390-ca80-41bb-acb5-dbdc9bb0547e admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:30.459958 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da98190a-d403-40dc-b82f-e1c597fc34ef admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:30.462951 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c112390-ca80-41bb-acb5-dbdc9bb0547e admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:30.465122 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ad8af2-e290-4e27-af3c-64c18d37d649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:30.465372 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7907216-003c-4a52-9fb4-ece0bb6561e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:30.465952 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ad8af2-e290-4e27-af3c-64c18d37d649 admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:30.466268 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ad8af2-e290-4e27-af3c-64c18d37d649 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:30.466375 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7907216-003c-4a52-9fb4-ece0bb6561e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:30.466568 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ad8af2-e290-4e27-af3c-64c18d37d649 admin admin] PATH_INFO: `/v3/projects/1f36d409ce8e41c09c65c8b244ddbba9/users/d65556a2bbd0485f97f0ab1cbd72a3d2/roles/65feabf3b27944a8b01a86c8ac608457` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:30.466672 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7907216-003c-4a52-9fb4-ece0bb6561e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:30.466871 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7907216-003c-4a52-9fb4-ece0bb6561e6 admin admin] PATH_INFO: `/v3/projects/86556ffcfd5b4d5399db96496f165dc4/users/29595bd142334fbd8b2aff1d7f7a3685/roles/65feabf3b27944a8b01a86c8ac608457` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:30.483079 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ad8af2-e290-4e27-af3c-64c18d37d649 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f36d409ce8e41c09c65c8b244ddbba9, user_id=d65556a2bbd0485f97f0ab1cbd72a3d2, role_id=65feabf3b27944a8b01a86c8ac608457)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:30.483287 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7907216-003c-4a52-9fb4-ece0bb6561e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=86556ffcfd5b4d5399db96496f165dc4, user_id=29595bd142334fbd8b2aff1d7f7a3685, role_id=65feabf3b27944a8b01a86c8ac608457)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:30.485776 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7907216-003c-4a52-9fb4-ece0bb6561e6 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:30.491612 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ad8af2-e290-4e27-af3c-64c18d37d649 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:30.566453 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1211/2453] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:42:30 2022] PUT /identity/v3/projects/86556ffcfd5b4d5399db96496f165dc4/users/29595bd142334fbd8b2aff1d7f7a3685/roles/65feabf3b27944a8b01a86c8ac608457 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:42:30.580753 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34326d01-06a3-40ad-870c-7c6b59238cf8 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:30.581061 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34326d01-06a3-40ad-870c-7c6b59238cf8 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:30.581314 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34326d01-06a3-40ad-870c-7c6b59238cf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:30.582512 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1243/2454] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:42:30 2022] PUT /identity/v3/projects/1f36d409ce8e41c09c65c8b244ddbba9/users/d65556a2bbd0485f97f0ab1cbd72a3d2/roles/65feabf3b27944a8b01a86c8ac608457 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:42:30.621600 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-34326d01-06a3-40ad-870c-7c6b59238cf8 None None] MFA Rules not processed for user `29595bd142334fbd8b2aff1d7f7a3685`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:42:30.623915 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-34326d01-06a3-40ad-870c-7c6b59238cf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:30.649685 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2e3c43-d6ff-4f54-838e-a1358ac8fdea None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:30.649988 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2e3c43-d6ff-4f54-838e-a1358ac8fdea None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:30.650303 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2e3c43-d6ff-4f54-838e-a1358ac8fdea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:30.670962 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:42:30.677483 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-2d2e3c43-d6ff-4f54-838e-a1358ac8fdea None None] MFA Rules not processed for user `d65556a2bbd0485f97f0ab1cbd72a3d2`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:42:30.678742 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-2d2e3c43-d6ff-4f54-838e-a1358ac8fdea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:30.700417 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1212/2455] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:42:30 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 02:42:30.704895 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34326d01-06a3-40ad-870c-7c6b59238cf8 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:30.784759 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1244/2456] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:42:30 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 02:42:30.788782 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2e3c43-d6ff-4f54-838e-a1358ac8fdea None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:30.797602 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34326d01-06a3-40ad-870c-7c6b59238cf8 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:30.800591 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2e3c43-d6ff-4f54-838e-a1358ac8fdea None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:30.803076 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3d542d-ead0-42b6-b2b0-45cd06ddc4f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:30.803834 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3d542d-ead0-42b6-b2b0-45cd06ddc4f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:30.804190 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3d542d-ead0-42b6-b2b0-45cd06ddc4f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:30.804551 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3d542d-ead0-42b6-b2b0-45cd06ddc4f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:30.805309 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c07e82-52e9-4006-bdc3-91cc63089ae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:30.805494 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3d542d-ead0-42b6-b2b0-45cd06ddc4f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:30.805957 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c07e82-52e9-4006-bdc3-91cc63089ae0 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:30.806291 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c07e82-52e9-4006-bdc3-91cc63089ae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:30.806576 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c07e82-52e9-4006-bdc3-91cc63089ae0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:30.807400 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c07e82-52e9-4006-bdc3-91cc63089ae0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:30.807765 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3d542d-ead0-42b6-b2b0-45cd06ddc4f2 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:30.809524 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c07e82-52e9-4006-bdc3-91cc63089ae0 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:30.814879 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:42:30.910502 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1213/2457] 10.222.0.66 () {60 vars in 1152 bytes} [Mon Jul 18 02:42:30 2022] POST /identity/v3/projects => generated 379 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:42:30.914433 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3d542d-ead0-42b6-b2b0-45cd06ddc4f2 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:30.970126 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1245/2458] 10.222.0.66 () {60 vars in 1152 bytes} [Mon Jul 18 02:42:30 2022] POST /identity/v3/projects => generated 377 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 02:42:30.971691 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c07e82-52e9-4006-bdc3-91cc63089ae0 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:30.980953 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3d542d-ead0-42b6-b2b0-45cd06ddc4f2 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:31.035801 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c07e82-52e9-4006-bdc3-91cc63089ae0 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:31.040092 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93285ba7-00ff-4611-8e1b-c64cf10d6cdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:31.040504 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecd70b7-9572-4cf3-8108-a2b31d00769d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:31.040767 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93285ba7-00ff-4611-8e1b-c64cf10d6cdc admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:31.041050 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93285ba7-00ff-4611-8e1b-c64cf10d6cdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:31.041164 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecd70b7-9572-4cf3-8108-a2b31d00769d admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:31.041327 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93285ba7-00ff-4611-8e1b-c64cf10d6cdc admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:31.041438 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecd70b7-9572-4cf3-8108-a2b31d00769d admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:31.041702 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecd70b7-9572-4cf3-8108-a2b31d00769d admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:31.042014 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93285ba7-00ff-4611-8e1b-c64cf10d6cdc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:31.042449 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecd70b7-9572-4cf3-8108-a2b31d00769d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:31.044052 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93285ba7-00ff-4611-8e1b-c64cf10d6cdc admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:31.044846 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecd70b7-9572-4cf3-8108-a2b31d00769d admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:31.138851 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:42:31.143879 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1214/2459] 10.222.0.66 () {60 vars in 1143 bytes} [Mon Jul 18 02:42:30 2022] POST /identity/v3/users => generated 290 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 02:42:31.147768 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93285ba7-00ff-4611-8e1b-c64cf10d6cdc admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:31.157403 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93285ba7-00ff-4611-8e1b-c64cf10d6cdc admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:31.162041 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f130fd62-7ddd-4f4f-b5f4-159bf2051741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:31.162704 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f130fd62-7ddd-4f4f-b5f4-159bf2051741 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:31.163013 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f130fd62-7ddd-4f4f-b5f4-159bf2051741 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:31.163292 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f130fd62-7ddd-4f4f-b5f4-159bf2051741 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:31.164010 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f130fd62-7ddd-4f4f-b5f4-159bf2051741 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:31.165596 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f130fd62-7ddd-4f4f-b5f4-159bf2051741 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:31.175390 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1215/2460] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:42:31 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 02:42:31.178998 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f130fd62-7ddd-4f4f-b5f4-159bf2051741 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:31.188857 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f130fd62-7ddd-4f4f-b5f4-159bf2051741 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:31.193471 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ce45bf-4116-44ba-9306-ec97c2f20ef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:31.194065 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ce45bf-4116-44ba-9306-ec97c2f20ef7 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:31.194383 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ce45bf-4116-44ba-9306-ec97c2f20ef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:31.194661 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ce45bf-4116-44ba-9306-ec97c2f20ef7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:31.195376 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ce45bf-4116-44ba-9306-ec97c2f20ef7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:31.196891 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ce45bf-4116-44ba-9306-ec97c2f20ef7 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:31.206561 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1246/2461] 10.222.0.66 () {60 vars in 1143 bytes} [Mon Jul 18 02:42:30 2022] POST /identity/v3/users => generated 289 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 02:42:31.211786 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1216/2462] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:42:31 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 02:42:31.212602 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecd70b7-9572-4cf3-8108-a2b31d00769d admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:31.212728 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ce45bf-4116-44ba-9306-ec97c2f20ef7 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:31.222468 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ce45bf-4116-44ba-9306-ec97c2f20ef7 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:31.222704 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecd70b7-9572-4cf3-8108-a2b31d00769d admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:31.227293 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f87fdc-6335-4821-be0c-73625737f4da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:31.227492 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf1c4c6-0578-40af-a3a2-e8ca862ac547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:31.228143 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf1c4c6-0578-40af-a3a2-e8ca862ac547 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:31.228243 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f87fdc-6335-4821-be0c-73625737f4da admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:31.228429 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f87fdc-6335-4821-be0c-73625737f4da admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:31.228527 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf1c4c6-0578-40af-a3a2-e8ca862ac547 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:31.228708 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf1c4c6-0578-40af-a3a2-e8ca862ac547 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:31.228802 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f87fdc-6335-4821-be0c-73625737f4da admin admin] PATH_INFO: `/v3/projects/8cedaff70e0746f9b7f655e71184a494/users/0f27c74afc6f439a9b436a834e19c48d/roles/65feabf3b27944a8b01a86c8ac608457` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:31.229497 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf1c4c6-0578-40af-a3a2-e8ca862ac547 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:31.231807 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf1c4c6-0578-40af-a3a2-e8ca862ac547 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:31.242469 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1247/2463] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:42:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:42:31.244833 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f87fdc-6335-4821-be0c-73625737f4da admin admin] RBAC: Authorizing `identity:create_grant(project_id=8cedaff70e0746f9b7f655e71184a494, user_id=0f27c74afc6f439a9b436a834e19c48d, role_id=65feabf3b27944a8b01a86c8ac608457)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:31.247115 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf1c4c6-0578-40af-a3a2-e8ca862ac547 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:31.248381 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f87fdc-6335-4821-be0c-73625737f4da admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:31.256899 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf1c4c6-0578-40af-a3a2-e8ca862ac547 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:31.261461 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54b4083-98cf-4675-b742-43504905e258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:31.262140 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54b4083-98cf-4675-b742-43504905e258 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:31.262451 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54b4083-98cf-4675-b742-43504905e258 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:31.262697 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54b4083-98cf-4675-b742-43504905e258 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:31.263543 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54b4083-98cf-4675-b742-43504905e258 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:31.265124 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54b4083-98cf-4675-b742-43504905e258 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:31.275863 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1248/2464] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:42:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 02:42:31.280190 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54b4083-98cf-4675-b742-43504905e258 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:31.290134 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54b4083-98cf-4675-b742-43504905e258 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:31.295608 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960aa879-19d2-4bd4-aea5-7c6144a2307b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:31.296468 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960aa879-19d2-4bd4-aea5-7c6144a2307b admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:31.296748 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960aa879-19d2-4bd4-aea5-7c6144a2307b admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:31.297019 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960aa879-19d2-4bd4-aea5-7c6144a2307b admin admin] PATH_INFO: `/v3/projects/b4150e808c9443c8ac7652a31b5cce5e/users/7e55c6386a794005989a9abbb75e2b4a/roles/65feabf3b27944a8b01a86c8ac608457` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:31.312721 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960aa879-19d2-4bd4-aea5-7c6144a2307b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4150e808c9443c8ac7652a31b5cce5e, user_id=7e55c6386a794005989a9abbb75e2b4a, role_id=65feabf3b27944a8b01a86c8ac608457)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:31.315430 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960aa879-19d2-4bd4-aea5-7c6144a2307b admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:31.366049 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1217/2465] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:42:31 2022] PUT /identity/v3/projects/8cedaff70e0746f9b7f655e71184a494/users/0f27c74afc6f439a9b436a834e19c48d/roles/65feabf3b27944a8b01a86c8ac608457 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 02:42:31.378201 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4743c0c-6fce-4204-a57f-eb5945663018 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:31.378543 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4743c0c-6fce-4204-a57f-eb5945663018 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:31.378850 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4743c0c-6fce-4204-a57f-eb5945663018 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:31.405666 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-b4743c0c-6fce-4204-a57f-eb5945663018 None None] MFA Rules not processed for user `0f27c74afc6f439a9b436a834e19c48d`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:42:31.406947 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-b4743c0c-6fce-4204-a57f-eb5945663018 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:31.450178 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1249/2466] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:42:31 2022] PUT /identity/v3/projects/b4150e808c9443c8ac7652a31b5cce5e/users/7e55c6386a794005989a9abbb75e2b4a/roles/65feabf3b27944a8b01a86c8ac608457 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:42:31.460314 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4202a0-6d69-45ff-b9eb-9bf1b43b2427 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:31.460610 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4202a0-6d69-45ff-b9eb-9bf1b43b2427 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:31.460895 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4202a0-6d69-45ff-b9eb-9bf1b43b2427 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:31.472765 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1218/2467] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:42:31 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 02:42:31.478586 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4743c0c-6fce-4204-a57f-eb5945663018 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:31.489721 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-7c4202a0-6d69-45ff-b9eb-9bf1b43b2427 None None] MFA Rules not processed for user `7e55c6386a794005989a9abbb75e2b4a`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:42:31.491054 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-7c4202a0-6d69-45ff-b9eb-9bf1b43b2427 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:31.543494 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4743c0c-6fce-4204-a57f-eb5945663018 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:31.557565 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1250/2468] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:42:31 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:42:31.563027 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4202a0-6d69-45ff-b9eb-9bf1b43b2427 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:31.584054 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:42:31.596624 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:42:31.607060 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e213ecb5-7fd2-4e5c-8ca3-f22e73114a5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:31.609473 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e213ecb5-7fd2-4e5c-8ca3-f22e73114a5b admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:31.609473 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e213ecb5-7fd2-4e5c-8ca3-f22e73114a5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:31.609473 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e213ecb5-7fd2-4e5c-8ca3-f22e73114a5b admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:31.609473 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e213ecb5-7fd2-4e5c-8ca3-f22e73114a5b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:31.610973 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e213ecb5-7fd2-4e5c-8ca3-f22e73114a5b admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:31.637413 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4202a0-6d69-45ff-b9eb-9bf1b43b2427 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:31.642152 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4c40ec-0cdb-4844-bad5-ff9b878788be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:31.642810 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4c40ec-0cdb-4844-bad5-ff9b878788be admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:31.643093 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4c40ec-0cdb-4844-bad5-ff9b878788be admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:31.643371 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4c40ec-0cdb-4844-bad5-ff9b878788be admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:31.644060 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4c40ec-0cdb-4844-bad5-ff9b878788be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:31.646022 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4c40ec-0cdb-4844-bad5-ff9b878788be admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:31.755890 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1219/2469] 10.222.0.66 () {60 vars in 1152 bytes} [Mon Jul 18 02:42:31 2022] POST /identity/v3/projects => generated 377 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 02:42:31.760190 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e213ecb5-7fd2-4e5c-8ca3-f22e73114a5b admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:31.792030 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1251/2470] 10.222.0.66 () {60 vars in 1152 bytes} [Mon Jul 18 02:42:31 2022] POST /identity/v3/projects => generated 377 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 02:42:31.796670 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4c40ec-0cdb-4844-bad5-ff9b878788be admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:31.831606 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e213ecb5-7fd2-4e5c-8ca3-f22e73114a5b admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:31.866855 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4c40ec-0cdb-4844-bad5-ff9b878788be admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:31.871655 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cf083b-fc9c-4a43-be13-904c34193840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:31.872326 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cf083b-fc9c-4a43-be13-904c34193840 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:31.872615 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cf083b-fc9c-4a43-be13-904c34193840 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:31.872915 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cf083b-fc9c-4a43-be13-904c34193840 admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:31.873591 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cf083b-fc9c-4a43-be13-904c34193840 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:31.875596 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cf083b-fc9c-4a43-be13-904c34193840 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:31.893454 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfffcc97-f7d5-4686-8a3f-a8c4344749c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:31.896128 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfffcc97-f7d5-4686-8a3f-a8c4344749c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:31.896128 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfffcc97-f7d5-4686-8a3f-a8c4344749c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:31.896128 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfffcc97-f7d5-4686-8a3f-a8c4344749c9 admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:31.896128 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfffcc97-f7d5-4686-8a3f-a8c4344749c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:31.897141 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfffcc97-f7d5-4686-8a3f-a8c4344749c9 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:31.989038 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1252/2471] 10.222.0.66 () {60 vars in 1143 bytes} [Mon Jul 18 02:42:31 2022] POST /identity/v3/users => generated 295 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 02:42:31.993209 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cf083b-fc9c-4a43-be13-904c34193840 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:32.001816 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1220/2472] 10.222.0.66 () {60 vars in 1143 bytes} [Mon Jul 18 02:42:31 2022] POST /identity/v3/users => generated 295 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 02:42:32.002972 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfffcc97-f7d5-4686-8a3f-a8c4344749c9 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:32.007104 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cf083b-fc9c-4a43-be13-904c34193840 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:32.012850 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c2f8ee-5ef1-4b7d-bcc2-99720455196d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:32.013627 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c2f8ee-5ef1-4b7d-bcc2-99720455196d admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:32.017374 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c2f8ee-5ef1-4b7d-bcc2-99720455196d admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:32.017374 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c2f8ee-5ef1-4b7d-bcc2-99720455196d admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:32.017374 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c2f8ee-5ef1-4b7d-bcc2-99720455196d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:32.017685 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfffcc97-f7d5-4686-8a3f-a8c4344749c9 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:32.017788 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c2f8ee-5ef1-4b7d-bcc2-99720455196d admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:32.023724 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f7e7d5-f31b-4345-be61-03afc474e1a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:32.024583 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f7e7d5-f31b-4345-be61-03afc474e1a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:32.024943 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f7e7d5-f31b-4345-be61-03afc474e1a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:32.025370 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f7e7d5-f31b-4345-be61-03afc474e1a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:32.026169 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f7e7d5-f31b-4345-be61-03afc474e1a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:32.028650 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f7e7d5-f31b-4345-be61-03afc474e1a2 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:32.029763 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1253/2473] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:42:31 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:42:32.035071 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c2f8ee-5ef1-4b7d-bcc2-99720455196d admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:32.045882 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c2f8ee-5ef1-4b7d-bcc2-99720455196d admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:32.046468 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1221/2474] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:42:32 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:42:32.048211 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f7e7d5-f31b-4345-be61-03afc474e1a2 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:32.051886 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae4f250-cb39-4986-af6c-5af49c07ce98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:32.052108 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae4f250-cb39-4986-af6c-5af49c07ce98 admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:32.052500 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae4f250-cb39-4986-af6c-5af49c07ce98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:32.052784 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae4f250-cb39-4986-af6c-5af49c07ce98 admin admin] PATH_INFO: `/v3/projects/2815332cd1d3486291843a8af1aa95ee/users/cb4fbc5b4f814e10b9c61968e4fbc253/roles/d4aa3324f0c84887b41d0dc4acf7f30e` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:32.058480 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f7e7d5-f31b-4345-be61-03afc474e1a2 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:32.063634 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8557b1b-eb3f-4db9-81e2-f8e5d827d261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:32.064498 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8557b1b-eb3f-4db9-81e2-f8e5d827d261 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:32.064773 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8557b1b-eb3f-4db9-81e2-f8e5d827d261 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:32.065042 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8557b1b-eb3f-4db9-81e2-f8e5d827d261 admin admin] PATH_INFO: `/v3/projects/31a1e3123ffe4fd6bf727e266b8fbd78/users/3001201b05ca4aa89f54fd282b5d65ee/roles/d4aa3324f0c84887b41d0dc4acf7f30e` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:32.075150 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae4f250-cb39-4986-af6c-5af49c07ce98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2815332cd1d3486291843a8af1aa95ee, user_id=cb4fbc5b4f814e10b9c61968e4fbc253, role_id=d4aa3324f0c84887b41d0dc4acf7f30e)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:32.080548 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae4f250-cb39-4986-af6c-5af49c07ce98 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:32.081004 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8557b1b-eb3f-4db9-81e2-f8e5d827d261 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31a1e3123ffe4fd6bf727e266b8fbd78, user_id=3001201b05ca4aa89f54fd282b5d65ee, role_id=d4aa3324f0c84887b41d0dc4acf7f30e)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:32.083760 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8557b1b-eb3f-4db9-81e2-f8e5d827d261 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:32.177000 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1254/2475] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:42:32 2022] PUT /identity/v3/projects/2815332cd1d3486291843a8af1aa95ee/users/cb4fbc5b4f814e10b9c61968e4fbc253/roles/d4aa3324f0c84887b41d0dc4acf7f30e => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:42:32.180853 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae4f250-cb39-4986-af6c-5af49c07ce98 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:32.189279 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1222/2476] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:42:32 2022] PUT /identity/v3/projects/31a1e3123ffe4fd6bf727e266b8fbd78/users/3001201b05ca4aa89f54fd282b5d65ee/roles/d4aa3324f0c84887b41d0dc4acf7f30e => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 02:42:32.193120 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8557b1b-eb3f-4db9-81e2-f8e5d827d261 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:32.234870 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:42:32.243964 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae4f250-cb39-4986-af6c-5af49c07ce98 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:32.248104 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:42:32.254986 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8557b1b-eb3f-4db9-81e2-f8e5d827d261 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:32.259610 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467dfa36-7ca6-47aa-bf4b-a563202de404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:32.260272 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467dfa36-7ca6-47aa-bf4b-a563202de404 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:32.260556 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467dfa36-7ca6-47aa-bf4b-a563202de404 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:32.260835 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467dfa36-7ca6-47aa-bf4b-a563202de404 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:32.262180 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467dfa36-7ca6-47aa-bf4b-a563202de404 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:32.264134 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467dfa36-7ca6-47aa-bf4b-a563202de404 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:32.274027 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1223/2477] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:42:32 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 02:42:32.279011 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467dfa36-7ca6-47aa-bf4b-a563202de404 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:32.291007 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467dfa36-7ca6-47aa-bf4b-a563202de404 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:32.295940 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3a37a6-8b02-4728-b038-e87df7df2c4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:32.296821 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3a37a6-8b02-4728-b038-e87df7df2c4f admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:32.299626 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3a37a6-8b02-4728-b038-e87df7df2c4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:32.299626 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3a37a6-8b02-4728-b038-e87df7df2c4f admin admin] PATH_INFO: `/v3/projects/31a1e3123ffe4fd6bf727e266b8fbd78/users/3001201b05ca4aa89f54fd282b5d65ee/roles/d4aa3324f0c84887b41d0dc4acf7f30e` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:32.299626 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3a37a6-8b02-4728-b038-e87df7df2c4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=31a1e3123ffe4fd6bf727e266b8fbd78, user_id=3001201b05ca4aa89f54fd282b5d65ee, role_id=d4aa3324f0c84887b41d0dc4acf7f30e)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:32.302146 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3a37a6-8b02-4728-b038-e87df7df2c4f admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:32.305817 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d27653c-b431-4426-b722-da76c9c3cfec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:32.306541 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d27653c-b431-4426-b722-da76c9c3cfec admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:32.306829 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d27653c-b431-4426-b722-da76c9c3cfec admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:32.307108 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d27653c-b431-4426-b722-da76c9c3cfec admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:32.307834 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d27653c-b431-4426-b722-da76c9c3cfec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:32.309686 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d27653c-b431-4426-b722-da76c9c3cfec admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:32.313177 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1224/2478] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:42:32 2022] PUT /identity/v3/projects/31a1e3123ffe4fd6bf727e266b8fbd78/users/3001201b05ca4aa89f54fd282b5d65ee/roles/d4aa3324f0c84887b41d0dc4acf7f30e => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 02:42:32.320541 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1255/2479] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:42:32 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 02:42:32.323455 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27799605-ed49-4367-b33a-a15adc8f2ed1 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:32.324261 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27799605-ed49-4367-b33a-a15adc8f2ed1 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:32.324844 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27799605-ed49-4367-b33a-a15adc8f2ed1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:32.325113 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d27653c-b431-4426-b722-da76c9c3cfec admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:32.335696 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d27653c-b431-4426-b722-da76c9c3cfec admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:32.351703 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-27799605-ed49-4367-b33a-a15adc8f2ed1 None None] MFA Rules not processed for user `3001201b05ca4aa89f54fd282b5d65ee`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:42:32.352912 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-27799605-ed49-4367-b33a-a15adc8f2ed1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:32.393110 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfa1fad-ab70-4e9a-a551-fa1f408f132a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:32.394003 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfa1fad-ab70-4e9a-a551-fa1f408f132a admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:32.394429 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfa1fad-ab70-4e9a-a551-fa1f408f132a admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:32.394707 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfa1fad-ab70-4e9a-a551-fa1f408f132a admin admin] PATH_INFO: `/v3/projects/2815332cd1d3486291843a8af1aa95ee/users/cb4fbc5b4f814e10b9c61968e4fbc253/roles/d4aa3324f0c84887b41d0dc4acf7f30e` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:32.396750 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfa1fad-ab70-4e9a-a551-fa1f408f132a admin admin] RBAC: Authorizing `identity:create_grant(project_id=2815332cd1d3486291843a8af1aa95ee, user_id=cb4fbc5b4f814e10b9c61968e4fbc253, role_id=d4aa3324f0c84887b41d0dc4acf7f30e)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:32.399522 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfa1fad-ab70-4e9a-a551-fa1f408f132a admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:32.410580 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1256/2480] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:42:32 2022] PUT /identity/v3/projects/2815332cd1d3486291843a8af1aa95ee/users/cb4fbc5b4f814e10b9c61968e4fbc253/roles/d4aa3324f0c84887b41d0dc4acf7f30e => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 02:42:32.422393 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709d928c-8c91-45d9-8386-0ffd046faf69 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:32.425715 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709d928c-8c91-45d9-8386-0ffd046faf69 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:32.425715 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709d928c-8c91-45d9-8386-0ffd046faf69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:32.426217 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1225/2481] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:42:32 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 02:42:32.438466 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27799605-ed49-4367-b33a-a15adc8f2ed1 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:32.440054 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-27799605-ed49-4367-b33a-a15adc8f2ed1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:32.452810 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-709d928c-8c91-45d9-8386-0ffd046faf69 None None] MFA Rules not processed for user `cb4fbc5b4f814e10b9c61968e4fbc253`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:42:32.454020 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-709d928c-8c91-45d9-8386-0ffd046faf69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:32.499059 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27799605-ed49-4367-b33a-a15adc8f2ed1 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:32.527564 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1257/2482] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:42:32 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 02:42:32.539198 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709d928c-8c91-45d9-8386-0ffd046faf69 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:32.540583 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-709d928c-8c91-45d9-8386-0ffd046faf69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:32.552209 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b395ec37-fe04-40fa-ae5e-460701cf80e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd49388facdf446885e55b45ef91f01e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:32.552862 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b395ec37-fe04-40fa-ae5e-460701cf80e6 service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:32.553143 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b395ec37-fe04-40fa-ae5e-460701cf80e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:32.553415 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b395ec37-fe04-40fa-ae5e-460701cf80e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:32.566871 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:42:32.568135 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:42:32.600328 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709d928c-8c91-45d9-8386-0ffd046faf69 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:32.604346 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a76b8be-3eac-4d72-9ef9-fae2d001600b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd49388facdf446885e55b45ef91f01e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:32.605169 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a76b8be-3eac-4d72-9ef9-fae2d001600b service cinder] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:32.605457 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a76b8be-3eac-4d72-9ef9-fae2d001600b service cinder] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:32.605740 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a76b8be-3eac-4d72-9ef9-fae2d001600b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:32.621358 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b395ec37-fe04-40fa-ae5e-460701cf80e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:32.623060 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b395ec37-fe04-40fa-ae5e-460701cf80e6 service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:32.632954 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1226/2483] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:42:32 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 02:42:32.666998 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a76b8be-3eac-4d72-9ef9-fae2d001600b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:32.668626 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a76b8be-3eac-4d72-9ef9-fae2d001600b service cinder] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:32.677958 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1258/2484] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:42:32 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 02:42:32.767018 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b395ec37-fe04-40fa-ae5e-460701cf80e6 service cinder] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:32.777472 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b395ec37-fe04-40fa-ae5e-460701cf80e6 service cinder] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:32.781857 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c734eded-d60b-47f0-be28-915f43aa729e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd49388facdf446885e55b45ef91f01e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:32.782284 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c734eded-d60b-47f0-be28-915f43aa729e service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:32.782570 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c734eded-d60b-47f0-be28-915f43aa729e service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:32.783612 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c734eded-d60b-47f0-be28-915f43aa729e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:32.844405 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c734eded-d60b-47f0-be28-915f43aa729e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:32.845836 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c734eded-d60b-47f0-be28-915f43aa729e service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:32.855511 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1227/2485] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:42:32 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:42:34.828656 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a76b8be-3eac-4d72-9ef9-fae2d001600b service cinder] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:34.837813 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a76b8be-3eac-4d72-9ef9-fae2d001600b service cinder] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:34.842309 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efffdeac-3ef5-46b1-9940-0e0bffdef760 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd49388facdf446885e55b45ef91f01e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:34.843112 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efffdeac-3ef5-46b1-9940-0e0bffdef760 service cinder] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:34.843400 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efffdeac-3ef5-46b1-9940-0e0bffdef760 service cinder] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:34.843680 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efffdeac-3ef5-46b1-9940-0e0bffdef760 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:34.914438 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efffdeac-3ef5-46b1-9940-0e0bffdef760 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:34.915928 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efffdeac-3ef5-46b1-9940-0e0bffdef760 service cinder] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:34.926403 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1259/2486] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:42:34 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:42:36.803812 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c734eded-d60b-47f0-be28-915f43aa729e service cinder] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:36.813760 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c734eded-d60b-47f0-be28-915f43aa729e service cinder] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:36.818921 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b331e6b-cad8-4f4f-93a4-fca685f22d6e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd49388facdf446885e55b45ef91f01e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:36.819632 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b331e6b-cad8-4f4f-93a4-fca685f22d6e service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:36.819979 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b331e6b-cad8-4f4f-93a4-fca685f22d6e service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:36.821244 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b331e6b-cad8-4f4f-93a4-fca685f22d6e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:36.830792 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b331e6b-cad8-4f4f-93a4-fca685f22d6e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:36.832213 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b331e6b-cad8-4f4f-93a4-fca685f22d6e service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:36.843042 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1228/2487] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:42:36 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:42:37.611383 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efffdeac-3ef5-46b1-9940-0e0bffdef760 service cinder] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:37.677982 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efffdeac-3ef5-46b1-9940-0e0bffdef760 service cinder] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:37.682881 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af106c00-d168-4254-9f94-2b796ad0c063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:37.683756 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af106c00-d168-4254-9f94-2b796ad0c063 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:37.684092 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af106c00-d168-4254-9f94-2b796ad0c063 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:37.684364 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af106c00-d168-4254-9f94-2b796ad0c063 admin admin] PATH_INFO: `/v3/users/29595bd142334fbd8b2aff1d7f7a3685` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:37.685622 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af106c00-d168-4254-9f94-2b796ad0c063 admin admin] RBAC: Authorizing `identity:delete_user(user_id=29595bd142334fbd8b2aff1d7f7a3685)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:37.687861 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af106c00-d168-4254-9f94-2b796ad0c063 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:37.703962 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:42:37.935716 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-af106c00-d168-4254-9f94-2b796ad0c063 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29595bd142334fbd8b2aff1d7f7a3685'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:37.994447 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-af106c00-d168-4254-9f94-2b796ad0c063 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29595bd142334fbd8b2aff1d7f7a3685'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:38.002356 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-af106c00-d168-4254-9f94-2b796ad0c063 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29595bd142334fbd8b2aff1d7f7a3685'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:38.003047 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-af106c00-d168-4254-9f94-2b796ad0c063 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29595bd142334fbd8b2aff1d7f7a3685'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:38.015809 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-af106c00-d168-4254-9f94-2b796ad0c063 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29595bd142334fbd8b2aff1d7f7a3685'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:38.021838 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1260/2488] 10.222.0.66 () {58 vars in 1223 bytes} [Mon Jul 18 02:42:37 2022] DELETE /identity/v3/users/29595bd142334fbd8b2aff1d7f7a3685 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:42:38.026546 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c62f6d-8d58-4aa0-a77b-f4e7bebef70d None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:38.027910 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c62f6d-8d58-4aa0-a77b-f4e7bebef70d None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:38.027910 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c62f6d-8d58-4aa0-a77b-f4e7bebef70d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:38.045223 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-75c62f6d-8d58-4aa0-a77b-f4e7bebef70d None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:42:38.046465 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-75c62f6d-8d58-4aa0-a77b-f4e7bebef70d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:38.106052 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1229/2489] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:42:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:42:38.115962 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af106c00-d168-4254-9f94-2b796ad0c063 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:38.117383 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-af106c00-d168-4254-9f94-2b796ad0c063 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:38.188484 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af106c00-d168-4254-9f94-2b796ad0c063 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:38.192078 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a95a30-4e12-4c60-aa7c-d0b5f6566699 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eaf19a259c2741cea375a38763e33cdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:38.192789 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a95a30-4e12-4c60-aa7c-d0b5f6566699 service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:38.193008 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a95a30-4e12-4c60-aa7c-d0b5f6566699 service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:38.193285 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a95a30-4e12-4c60-aa7c-d0b5f6566699 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:38.203417 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a95a30-4e12-4c60-aa7c-d0b5f6566699 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:38.204787 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a95a30-4e12-4c60-aa7c-d0b5f6566699 service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:38.215744 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1261/2490] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:42:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:42:38.648570 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c62f6d-8d58-4aa0-a77b-f4e7bebef70d None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:38.649998 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-75c62f6d-8d58-4aa0-a77b-f4e7bebef70d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:38.663623 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c62f6d-8d58-4aa0-a77b-f4e7bebef70d None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:38.669404 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac12a2f-a178-46a4-89d4-24d7a5452d79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:38.670384 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac12a2f-a178-46a4-89d4-24d7a5452d79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:38.670667 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac12a2f-a178-46a4-89d4-24d7a5452d79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:38.670941 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac12a2f-a178-46a4-89d4-24d7a5452d79 admin admin] PATH_INFO: `/v3/projects/86556ffcfd5b4d5399db96496f165dc4` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:38.672142 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac12a2f-a178-46a4-89d4-24d7a5452d79 admin admin] RBAC: Authorizing `identity:delete_project(project_id=86556ffcfd5b4d5399db96496f165dc4)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:38.674069 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac12a2f-a178-46a4-89d4-24d7a5452d79 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:38.819122 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:42:38.833325 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-3ac12a2f-a178-46a4-89d4-24d7a5452d79 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86556ffcfd5b4d5399db96496f165dc4', 'request_id': 'req-3ac12a2f-a178-46a4-89d4-24d7a5452d79'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:38.839978 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-3ac12a2f-a178-46a4-89d4-24d7a5452d79 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86556ffcfd5b4d5399db96496f165dc4', 'request_id': 'req-3ac12a2f-a178-46a4-89d4-24d7a5452d79'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:38.846811 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-3ac12a2f-a178-46a4-89d4-24d7a5452d79 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86556ffcfd5b4d5399db96496f165dc4', 'request_id': 'req-3ac12a2f-a178-46a4-89d4-24d7a5452d79'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:38.912474 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-3ac12a2f-a178-46a4-89d4-24d7a5452d79 admin admin] The token cache is being invalidate because project 86556ffcfd5b4d5399db96496f165dc4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 02:42:38.912794 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-3ac12a2f-a178-46a4-89d4-24d7a5452d79 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:38.913448 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-3ac12a2f-a178-46a4-89d4-24d7a5452d79 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:38.915581 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1230/2491] 10.222.0.66 () {58 vars in 1232 bytes} [Mon Jul 18 02:42:38 2022] DELETE /identity/v3/projects/86556ffcfd5b4d5399db96496f165dc4 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:42:38.924855 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a95a30-4e12-4c60-aa7c-d0b5f6566699 service neutron] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:38.935543 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a95a30-4e12-4c60-aa7c-d0b5f6566699 service neutron] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:39.001277 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eded2c23-8ccb-4a95-a7ea-55a28d910fac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:39.001687 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eded2c23-8ccb-4a95-a7ea-55a28d910fac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:39.002000 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eded2c23-8ccb-4a95-a7ea-55a28d910fac admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:39.002321 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eded2c23-8ccb-4a95-a7ea-55a28d910fac admin admin] PATH_INFO: `/v3/users/0f27c74afc6f439a9b436a834e19c48d` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:39.006227 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eded2c23-8ccb-4a95-a7ea-55a28d910fac admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f27c74afc6f439a9b436a834e19c48d)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:39.006227 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eded2c23-8ccb-4a95-a7ea-55a28d910fac admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:39.121975 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac12a2f-a178-46a4-89d4-24d7a5452d79 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:39.123401 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-3ac12a2f-a178-46a4-89d4-24d7a5452d79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:39.136543 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac12a2f-a178-46a4-89d4-24d7a5452d79 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:39.141424 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654d4363-df9d-4ee9-a5bb-61d657438878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:39.142326 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654d4363-df9d-4ee9-a5bb-61d657438878 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:39.142604 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654d4363-df9d-4ee9-a5bb-61d657438878 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:39.142878 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654d4363-df9d-4ee9-a5bb-61d657438878 admin admin] PATH_INFO: `/v3/users/1c9f18acfaab426abc52542ff5ddbee9` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:39.144052 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654d4363-df9d-4ee9-a5bb-61d657438878 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c9f18acfaab426abc52542ff5ddbee9)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:39.146113 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654d4363-df9d-4ee9-a5bb-61d657438878 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:39.219176 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:42:39.225106 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-eded2c23-8ccb-4a95-a7ea-55a28d910fac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f27c74afc6f439a9b436a834e19c48d'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:39.303163 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-eded2c23-8ccb-4a95-a7ea-55a28d910fac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f27c74afc6f439a9b436a834e19c48d'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:39.313711 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-eded2c23-8ccb-4a95-a7ea-55a28d910fac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f27c74afc6f439a9b436a834e19c48d'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:39.313711 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-eded2c23-8ccb-4a95-a7ea-55a28d910fac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f27c74afc6f439a9b436a834e19c48d'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:39.323124 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-eded2c23-8ccb-4a95-a7ea-55a28d910fac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f27c74afc6f439a9b436a834e19c48d'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:39.328993 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1262/2492] 10.222.0.66 () {58 vars in 1223 bytes} [Mon Jul 18 02:42:38 2022] DELETE /identity/v3/users/0f27c74afc6f439a9b436a834e19c48d => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:42:39.332209 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eded2c23-8ccb-4a95-a7ea-55a28d910fac admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:39.332209 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-eded2c23-8ccb-4a95-a7ea-55a28d910fac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:39.353870 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-654d4363-df9d-4ee9-a5bb-61d657438878 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c9f18acfaab426abc52542ff5ddbee9'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:39.393483 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eded2c23-8ccb-4a95-a7ea-55a28d910fac admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:39.397144 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaabe1c-2c3c-4e55-bfab-a287be41584e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eaf19a259c2741cea375a38763e33cdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:39.397793 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaabe1c-2c3c-4e55-bfab-a287be41584e service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:39.398103 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaabe1c-2c3c-4e55-bfab-a287be41584e service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:39.398436 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaabe1c-2c3c-4e55-bfab-a287be41584e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:39.400231 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-cfaabe1c-2c3c-4e55-bfab-a287be41584e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:39.444389 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-654d4363-df9d-4ee9-a5bb-61d657438878 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c9f18acfaab426abc52542ff5ddbee9'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:39.445049 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-654d4363-df9d-4ee9-a5bb-61d657438878 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c9f18acfaab426abc52542ff5ddbee9'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:39.459158 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-654d4363-df9d-4ee9-a5bb-61d657438878 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c9f18acfaab426abc52542ff5ddbee9'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:39.460119 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-654d4363-df9d-4ee9-a5bb-61d657438878 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c9f18acfaab426abc52542ff5ddbee9'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:39.467859 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaabe1c-2c3c-4e55-bfab-a287be41584e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:39.469321 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaabe1c-2c3c-4e55-bfab-a287be41584e service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:39.474375 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1231/2493] 10.222.0.66 () {58 vars in 1223 bytes} [Mon Jul 18 02:42:39 2022] DELETE /identity/v3/users/1c9f18acfaab426abc52542ff5ddbee9 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 02:42:39.478752 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eb4918-7297-46c1-8c4b-5ff45957df21 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:39.479099 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eb4918-7297-46c1-8c4b-5ff45957df21 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:39.483851 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eb4918-7297-46c1-8c4b-5ff45957df21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:39.484001 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1263/2494] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:42:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:42:39.499154 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-33eb4918-7297-46c1-8c4b-5ff45957df21 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:42:39.500419 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-33eb4918-7297-46c1-8c4b-5ff45957df21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:39.564320 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1232/2495] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:42:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:42:39.584699 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaabe1c-2c3c-4e55-bfab-a287be41584e service neutron] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:39.586580 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-cfaabe1c-2c3c-4e55-bfab-a287be41584e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:39.600808 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaabe1c-2c3c-4e55-bfab-a287be41584e service neutron] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:39.667257 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8f05ce-86dd-4baf-999d-af60c644178e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eaf19a259c2741cea375a38763e33cdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:39.670918 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8f05ce-86dd-4baf-999d-af60c644178e service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:39.670918 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8f05ce-86dd-4baf-999d-af60c644178e service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:39.670918 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8f05ce-86dd-4baf-999d-af60c644178e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:39.679202 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8f05ce-86dd-4baf-999d-af60c644178e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:39.680663 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8f05ce-86dd-4baf-999d-af60c644178e service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:39.692299 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1264/2496] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:42:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:42:39.922554 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33eb4918-7297-46c1-8c4b-5ff45957df21 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:39.924255 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-33eb4918-7297-46c1-8c4b-5ff45957df21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:39.939513 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33eb4918-7297-46c1-8c4b-5ff45957df21 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:39.945484 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80371e72-5016-4f0d-9ec1-66c95dbe23af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:39.946520 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80371e72-5016-4f0d-9ec1-66c95dbe23af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:39.946889 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80371e72-5016-4f0d-9ec1-66c95dbe23af admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:39.947402 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80371e72-5016-4f0d-9ec1-66c95dbe23af admin admin] PATH_INFO: `/v3/projects/8cedaff70e0746f9b7f655e71184a494` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:39.948817 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80371e72-5016-4f0d-9ec1-66c95dbe23af admin admin] RBAC: Authorizing `identity:delete_project(project_id=8cedaff70e0746f9b7f655e71184a494)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:39.952573 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80371e72-5016-4f0d-9ec1-66c95dbe23af admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:40.117502 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:42:40.138569 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-80371e72-5016-4f0d-9ec1-66c95dbe23af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8cedaff70e0746f9b7f655e71184a494', 'request_id': 'req-80371e72-5016-4f0d-9ec1-66c95dbe23af'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:40.145862 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-80371e72-5016-4f0d-9ec1-66c95dbe23af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8cedaff70e0746f9b7f655e71184a494', 'request_id': 'req-80371e72-5016-4f0d-9ec1-66c95dbe23af'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:40.153611 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-80371e72-5016-4f0d-9ec1-66c95dbe23af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8cedaff70e0746f9b7f655e71184a494', 'request_id': 'req-80371e72-5016-4f0d-9ec1-66c95dbe23af'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:40.241765 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8f05ce-86dd-4baf-999d-af60c644178e service neutron] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:40.243310 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-9c8f05ce-86dd-4baf-999d-af60c644178e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:40.243507 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-80371e72-5016-4f0d-9ec1-66c95dbe23af admin admin] The token cache is being invalidate because project 8cedaff70e0746f9b7f655e71184a494 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 02:42:40.243950 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-80371e72-5016-4f0d-9ec1-66c95dbe23af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:40.244906 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-80371e72-5016-4f0d-9ec1-66c95dbe23af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:40.246684 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1233/2497] 10.222.0.66 () {58 vars in 1232 bytes} [Mon Jul 18 02:42:39 2022] DELETE /identity/v3/projects/8cedaff70e0746f9b7f655e71184a494 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:42:40.250347 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80371e72-5016-4f0d-9ec1-66c95dbe23af admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:40.252008 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-80371e72-5016-4f0d-9ec1-66c95dbe23af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:40.264993 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8f05ce-86dd-4baf-999d-af60c644178e service neutron] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:40.275596 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:42:40.338698 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4479f744-ff0d-40b2-bf05-cd931337752b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:40.341079 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4479f744-ff0d-40b2-bf05-cd931337752b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:40.346460 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4479f744-ff0d-40b2-bf05-cd931337752b admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:40.346460 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4479f744-ff0d-40b2-bf05-cd931337752b admin admin] PATH_INFO: `/v3/projects/f9cad813a9304bd897bd93dc2277368b` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:40.346460 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4479f744-ff0d-40b2-bf05-cd931337752b admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9cad813a9304bd897bd93dc2277368b)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:40.346460 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4479f744-ff0d-40b2-bf05-cd931337752b admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:40.388249 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80371e72-5016-4f0d-9ec1-66c95dbe23af admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:40.391213 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2332494-a1e2-4be6-a7e0-9067a71acbb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:40.392183 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2332494-a1e2-4be6-a7e0-9067a71acbb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:40.392442 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2332494-a1e2-4be6-a7e0-9067a71acbb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:40.392716 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2332494-a1e2-4be6-a7e0-9067a71acbb7 admin admin] PATH_INFO: `/v3/users/3001201b05ca4aa89f54fd282b5d65ee` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:40.394069 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2332494-a1e2-4be6-a7e0-9067a71acbb7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3001201b05ca4aa89f54fd282b5d65ee)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:40.396037 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2332494-a1e2-4be6-a7e0-9067a71acbb7 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:40.515033 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-4479f744-ff0d-40b2-bf05-cd931337752b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9cad813a9304bd897bd93dc2277368b', 'request_id': 'req-4479f744-ff0d-40b2-bf05-cd931337752b'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:40.520462 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-4479f744-ff0d-40b2-bf05-cd931337752b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9cad813a9304bd897bd93dc2277368b', 'request_id': 'req-4479f744-ff0d-40b2-bf05-cd931337752b'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:40.626876 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-4479f744-ff0d-40b2-bf05-cd931337752b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9cad813a9304bd897bd93dc2277368b', 'request_id': 'req-4479f744-ff0d-40b2-bf05-cd931337752b'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:40.639514 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-4479f744-ff0d-40b2-bf05-cd931337752b admin admin] The token cache is being invalidate because project f9cad813a9304bd897bd93dc2277368b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 02:42:40.639803 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-4479f744-ff0d-40b2-bf05-cd931337752b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:40.640432 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-4479f744-ff0d-40b2-bf05-cd931337752b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:40.642034 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1265/2498] 10.222.0.66 () {58 vars in 1232 bytes} [Mon Jul 18 02:42:40 2022] DELETE /identity/v3/projects/f9cad813a9304bd897bd93dc2277368b => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 02:42:40.645928 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4479f744-ff0d-40b2-bf05-cd931337752b admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:40.647328 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-4479f744-ff0d-40b2-bf05-cd931337752b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:40.677609 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-e2332494-a1e2-4be6-a7e0-9067a71acbb7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3001201b05ca4aa89f54fd282b5d65ee'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:40.715019 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4479f744-ff0d-40b2-bf05-cd931337752b admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:40.720154 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054934fe-06fb-47b3-8476-097596388418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:40.721085 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054934fe-06fb-47b3-8476-097596388418 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:40.721372 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054934fe-06fb-47b3-8476-097596388418 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:40.721649 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054934fe-06fb-47b3-8476-097596388418 admin admin] PATH_INFO: `/v3/users/123b275490644613a2228f3856aa8f05` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:40.722915 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054934fe-06fb-47b3-8476-097596388418 admin admin] RBAC: Authorizing `identity:delete_user(user_id=123b275490644613a2228f3856aa8f05)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:40.725184 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054934fe-06fb-47b3-8476-097596388418 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:41.061787 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-e2332494-a1e2-4be6-a7e0-9067a71acbb7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3001201b05ca4aa89f54fd282b5d65ee'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:41.062762 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-e2332494-a1e2-4be6-a7e0-9067a71acbb7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3001201b05ca4aa89f54fd282b5d65ee'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:41.075238 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-e2332494-a1e2-4be6-a7e0-9067a71acbb7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3001201b05ca4aa89f54fd282b5d65ee'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:41.075893 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-e2332494-a1e2-4be6-a7e0-9067a71acbb7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3001201b05ca4aa89f54fd282b5d65ee'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:41.088040 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1234/2499] 10.222.0.66 () {58 vars in 1223 bytes} [Mon Jul 18 02:42:40 2022] DELETE /identity/v3/users/3001201b05ca4aa89f54fd282b5d65ee => generated 0 bytes in 838 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 02:42:41.111887 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-054934fe-06fb-47b3-8476-097596388418 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '123b275490644613a2228f3856aa8f05'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:41.502355 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-054934fe-06fb-47b3-8476-097596388418 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '123b275490644613a2228f3856aa8f05'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:41.506365 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-054934fe-06fb-47b3-8476-097596388418 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '123b275490644613a2228f3856aa8f05'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:41.507521 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-054934fe-06fb-47b3-8476-097596388418 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '123b275490644613a2228f3856aa8f05'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:41.524737 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:42:41.528423 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-054934fe-06fb-47b3-8476-097596388418 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '123b275490644613a2228f3856aa8f05'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:41.539920 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1266/2500] 10.222.0.66 () {58 vars in 1223 bytes} [Mon Jul 18 02:42:40 2022] DELETE /identity/v3/users/123b275490644613a2228f3856aa8f05 => generated 0 bytes in 894 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 02:42:41.802910 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2332494-a1e2-4be6-a7e0-9067a71acbb7 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:41.804470 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-e2332494-a1e2-4be6-a7e0-9067a71acbb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:41.852387 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:42:41.879664 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2332494-a1e2-4be6-a7e0-9067a71acbb7 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:41.939063 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5213f2f5-2db8-4c06-aeae-16265ba53ab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:41.939944 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5213f2f5-2db8-4c06-aeae-16265ba53ab2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:41.940232 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5213f2f5-2db8-4c06-aeae-16265ba53ab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:41.940511 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5213f2f5-2db8-4c06-aeae-16265ba53ab2 admin admin] PATH_INFO: `/v3/users/d65556a2bbd0485f97f0ab1cbd72a3d2` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:41.941738 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5213f2f5-2db8-4c06-aeae-16265ba53ab2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d65556a2bbd0485f97f0ab1cbd72a3d2)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:41.943705 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5213f2f5-2db8-4c06-aeae-16265ba53ab2 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:42.160624 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054934fe-06fb-47b3-8476-097596388418 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:42.162123 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-054934fe-06fb-47b3-8476-097596388418 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:42.182703 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054934fe-06fb-47b3-8476-097596388418 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:42.189815 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94f1aaf-af2b-4005-a222-5859e127218f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:42.192000 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94f1aaf-af2b-4005-a222-5859e127218f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:42.192000 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94f1aaf-af2b-4005-a222-5859e127218f admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:42.192000 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94f1aaf-af2b-4005-a222-5859e127218f admin admin] PATH_INFO: `/v3/projects/31a1e3123ffe4fd6bf727e266b8fbd78` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:42.193515 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94f1aaf-af2b-4005-a222-5859e127218f admin admin] RBAC: Authorizing `identity:delete_project(project_id=31a1e3123ffe4fd6bf727e266b8fbd78)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:42.196438 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94f1aaf-af2b-4005-a222-5859e127218f admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:42.208286 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-5213f2f5-2db8-4c06-aeae-16265ba53ab2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd65556a2bbd0485f97f0ab1cbd72a3d2'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:42.317582 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-5213f2f5-2db8-4c06-aeae-16265ba53ab2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd65556a2bbd0485f97f0ab1cbd72a3d2'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:42.318301 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-5213f2f5-2db8-4c06-aeae-16265ba53ab2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd65556a2bbd0485f97f0ab1cbd72a3d2'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:42.331761 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-b94f1aaf-af2b-4005-a222-5859e127218f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31a1e3123ffe4fd6bf727e266b8fbd78', 'request_id': 'req-b94f1aaf-af2b-4005-a222-5859e127218f'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:42.333388 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-5213f2f5-2db8-4c06-aeae-16265ba53ab2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd65556a2bbd0485f97f0ab1cbd72a3d2'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:42.334332 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-5213f2f5-2db8-4c06-aeae-16265ba53ab2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd65556a2bbd0485f97f0ab1cbd72a3d2'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:42.338239 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-b94f1aaf-af2b-4005-a222-5859e127218f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31a1e3123ffe4fd6bf727e266b8fbd78', 'request_id': 'req-b94f1aaf-af2b-4005-a222-5859e127218f'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:42.349704 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1235/2501] 10.222.0.66 () {58 vars in 1223 bytes} [Mon Jul 18 02:42:41 2022] DELETE /identity/v3/users/d65556a2bbd0485f97f0ab1cbd72a3d2 => generated 0 bytes in 547 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:42:42.351426 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5213f2f5-2db8-4c06-aeae-16265ba53ab2 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:42.352255 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-5213f2f5-2db8-4c06-aeae-16265ba53ab2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:42.417210 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5213f2f5-2db8-4c06-aeae-16265ba53ab2 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:42.422882 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6a07c9-1500-479f-b7c6-41fd6685c5fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:42.425680 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6a07c9-1500-479f-b7c6-41fd6685c5fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:42.425680 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6a07c9-1500-479f-b7c6-41fd6685c5fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:42.425680 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6a07c9-1500-479f-b7c6-41fd6685c5fd admin admin] PATH_INFO: `/v3/projects/87d1ef70a8f94526a0eb1d3f4bc61c56` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:42.425680 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6a07c9-1500-479f-b7c6-41fd6685c5fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=87d1ef70a8f94526a0eb1d3f4bc61c56)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:42.429374 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6a07c9-1500-479f-b7c6-41fd6685c5fd admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:42.430244 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-b94f1aaf-af2b-4005-a222-5859e127218f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31a1e3123ffe4fd6bf727e266b8fbd78', 'request_id': 'req-b94f1aaf-af2b-4005-a222-5859e127218f'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:42.443759 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-b94f1aaf-af2b-4005-a222-5859e127218f admin admin] The token cache is being invalidate because project 31a1e3123ffe4fd6bf727e266b8fbd78 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 02:42:42.444079 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-b94f1aaf-af2b-4005-a222-5859e127218f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:42.444729 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-b94f1aaf-af2b-4005-a222-5859e127218f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:42.446382 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1267/2502] 10.222.0.66 () {58 vars in 1232 bytes} [Mon Jul 18 02:42:42 2022] DELETE /identity/v3/projects/31a1e3123ffe4fd6bf727e266b8fbd78 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 02:42:42.448762 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49726c8e-e683-4c5b-9ec1-20675f1c3c08 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:42.448762 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49726c8e-e683-4c5b-9ec1-20675f1c3c08 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:42.448762 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49726c8e-e683-4c5b-9ec1-20675f1c3c08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:42.467902 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-49726c8e-e683-4c5b-9ec1-20675f1c3c08 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:42:42.469150 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-49726c8e-e683-4c5b-9ec1-20675f1c3c08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:42.477400 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1268/2503] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:42:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:42:42.478885 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e5fac1-9d08-4a15-8a64-249ba48d5d0d None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:42.479175 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e5fac1-9d08-4a15-8a64-249ba48d5d0d None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:42.479457 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e5fac1-9d08-4a15-8a64-249ba48d5d0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:42.499683 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-60e5fac1-9d08-4a15-8a64-249ba48d5d0d None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:42:42.500952 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-60e5fac1-9d08-4a15-8a64-249ba48d5d0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:42.509284 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1269/2504] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:42:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:42:42.509996 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e5fac1-9d08-4a15-8a64-249ba48d5d0d None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:42.511302 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-60e5fac1-9d08-4a15-8a64-249ba48d5d0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:42.549019 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-3e6a07c9-1500-479f-b7c6-41fd6685c5fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87d1ef70a8f94526a0eb1d3f4bc61c56', 'request_id': 'req-3e6a07c9-1500-479f-b7c6-41fd6685c5fd'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:42.554016 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-3e6a07c9-1500-479f-b7c6-41fd6685c5fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87d1ef70a8f94526a0eb1d3f4bc61c56', 'request_id': 'req-3e6a07c9-1500-479f-b7c6-41fd6685c5fd'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:42.562779 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-3e6a07c9-1500-479f-b7c6-41fd6685c5fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87d1ef70a8f94526a0eb1d3f4bc61c56', 'request_id': 'req-3e6a07c9-1500-479f-b7c6-41fd6685c5fd'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:42.579928 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e5fac1-9d08-4a15-8a64-249ba48d5d0d None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:42.638632 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7618dd-7dfe-44af-bcd4-9768bb2d3e9f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eaf19a259c2741cea375a38763e33cdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:42.639441 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7618dd-7dfe-44af-bcd4-9768bb2d3e9f service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:42.639733 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7618dd-7dfe-44af-bcd4-9768bb2d3e9f service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:42.640011 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7618dd-7dfe-44af-bcd4-9768bb2d3e9f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:42.647505 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-3e6a07c9-1500-479f-b7c6-41fd6685c5fd admin admin] The token cache is being invalidate because project 87d1ef70a8f94526a0eb1d3f4bc61c56 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 02:42:42.647977 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-3e6a07c9-1500-479f-b7c6-41fd6685c5fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:42.648993 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-3e6a07c9-1500-479f-b7c6-41fd6685c5fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:42.650740 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1236/2505] 10.222.0.66 () {58 vars in 1232 bytes} [Mon Jul 18 02:42:42 2022] DELETE /identity/v3/projects/87d1ef70a8f94526a0eb1d3f4bc61c56 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:42:42.652256 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdb1a1c-e583-4bc0-a065-1ad2af46dc79 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:42.655538 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdb1a1c-e583-4bc0-a065-1ad2af46dc79 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:42.655943 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdb1a1c-e583-4bc0-a065-1ad2af46dc79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:42.679718 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-4fdb1a1c-e583-4bc0-a065-1ad2af46dc79 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:42:42.680937 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-4fdb1a1c-e583-4bc0-a065-1ad2af46dc79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:42.709062 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7618dd-7dfe-44af-bcd4-9768bb2d3e9f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:42.710168 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7618dd-7dfe-44af-bcd4-9768bb2d3e9f service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:42.721516 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1270/2506] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:42:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:42:42.722743 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d058f668-66e2-4937-a2c0-ab3bd18f2a77 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:42.723087 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d058f668-66e2-4937-a2c0-ab3bd18f2a77 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:42.723372 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d058f668-66e2-4937-a2c0-ab3bd18f2a77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:42.749578 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-d058f668-66e2-4937-a2c0-ab3bd18f2a77 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:42:42.750388 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1237/2507] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:42:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:42:42.754117 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-d058f668-66e2-4937-a2c0-ab3bd18f2a77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:42.754912 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdb1a1c-e583-4bc0-a065-1ad2af46dc79 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:42.766213 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1271/2508] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:42:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:42:42.766414 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdb1a1c-e583-4bc0-a065-1ad2af46dc79 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:42.774278 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491ab292-f89f-4bc1-8966-bb59240c89f1 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:42.774586 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491ab292-f89f-4bc1-8966-bb59240c89f1 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:42.774873 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491ab292-f89f-4bc1-8966-bb59240c89f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:42.776785 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a424ff3-a1e4-482e-b6eb-c2e3cf160f53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:42.778855 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a424ff3-a1e4-482e-b6eb-c2e3cf160f53 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:42.779309 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a424ff3-a1e4-482e-b6eb-c2e3cf160f53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:42.781370 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a424ff3-a1e4-482e-b6eb-c2e3cf160f53 admin admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:42.782164 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a424ff3-a1e4-482e-b6eb-c2e3cf160f53 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:42:42.784589 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a424ff3-a1e4-482e-b6eb-c2e3cf160f53 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:42.784589 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a424ff3-a1e4-482e-b6eb-c2e3cf160f53 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:42.795408 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-491ab292-f89f-4bc1-8966-bb59240c89f1 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:42:42.797304 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-491ab292-f89f-4bc1-8966-bb59240c89f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:42.803366 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1238/2509] 10.222.0.66 () {58 vars in 1152 bytes} [Mon Jul 18 02:42:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 02:42:42.809071 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a424ff3-a1e4-482e-b6eb-c2e3cf160f53 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:42.813325 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1272/2510] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:42:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 02:42:42.819170 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491ab292-f89f-4bc1-8966-bb59240c89f1 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:42.822586 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a424ff3-a1e4-482e-b6eb-c2e3cf160f53 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:42.829427 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b894987-6bbf-43e2-a982-44659cb03325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:42.829651 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491ab292-f89f-4bc1-8966-bb59240c89f1 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:42.831154 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b894987-6bbf-43e2-a982-44659cb03325 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:42.831154 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b894987-6bbf-43e2-a982-44659cb03325 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:42.831154 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b894987-6bbf-43e2-a982-44659cb03325 admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:42.832556 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b894987-6bbf-43e2-a982-44659cb03325 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:42.834618 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b894987-6bbf-43e2-a982-44659cb03325 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:42.834907 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198417cb-708d-45f6-84a9-8951d1ec5384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:42.835679 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198417cb-708d-45f6-84a9-8951d1ec5384 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:42.836098 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198417cb-708d-45f6-84a9-8951d1ec5384 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:42.836472 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198417cb-708d-45f6-84a9-8951d1ec5384 admin admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:42.837412 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198417cb-708d-45f6-84a9-8951d1ec5384 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:42:42.838071 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198417cb-708d-45f6-84a9-8951d1ec5384 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:42.842779 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198417cb-708d-45f6-84a9-8951d1ec5384 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:42.857522 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1273/2511] 10.222.0.66 () {58 vars in 1152 bytes} [Mon Jul 18 02:42:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:42:42.862539 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198417cb-708d-45f6-84a9-8951d1ec5384 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:42.876433 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198417cb-708d-45f6-84a9-8951d1ec5384 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:42.880219 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed59ce1-8774-4a81-933d-3b8364725c67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:42.880972 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed59ce1-8774-4a81-933d-3b8364725c67 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:42.881359 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed59ce1-8774-4a81-933d-3b8364725c67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:42.881725 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed59ce1-8774-4a81-933d-3b8364725c67 admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:42.883009 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed59ce1-8774-4a81-933d-3b8364725c67 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:42.884452 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed59ce1-8774-4a81-933d-3b8364725c67 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:42.972680 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1239/2512] 10.222.0.66 () {60 vars in 1152 bytes} [Mon Jul 18 02:42:42 2022] POST /identity/v3/projects => generated 371 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:42:42.972680 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b894987-6bbf-43e2-a982-44659cb03325 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:42.989442 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:42:43.000461 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1274/2513] 10.222.0.66 () {60 vars in 1152 bytes} [Mon Jul 18 02:42:42 2022] POST /identity/v3/projects => generated 367 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 02:42:43.004137 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed59ce1-8774-4a81-933d-3b8364725c67 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:43.116563 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b894987-6bbf-43e2-a982-44659cb03325 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:43.144741 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed59ce1-8774-4a81-933d-3b8364725c67 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:43.153344 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db95d167-6480-47ea-8858-3a8c2b0989e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:43.154305 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db95d167-6480-47ea-8858-3a8c2b0989e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:43.154774 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db95d167-6480-47ea-8858-3a8c2b0989e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:43.155152 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db95d167-6480-47ea-8858-3a8c2b0989e4 admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:43.155960 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db95d167-6480-47ea-8858-3a8c2b0989e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:43.160007 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db95d167-6480-47ea-8858-3a8c2b0989e4 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:43.178210 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:42:43.221627 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f987b1b6-0da6-4b9b-ab47-50b204052805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:43.222341 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f987b1b6-0da6-4b9b-ab47-50b204052805 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:43.222627 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f987b1b6-0da6-4b9b-ab47-50b204052805 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:43.222938 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f987b1b6-0da6-4b9b-ab47-50b204052805 admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:43.223690 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f987b1b6-0da6-4b9b-ab47-50b204052805 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:43.225606 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f987b1b6-0da6-4b9b-ab47-50b204052805 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:43.265057 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1275/2514] 10.222.0.66 () {60 vars in 1143 bytes} [Mon Jul 18 02:42:43 2022] POST /identity/v3/users => generated 284 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:42:43.270353 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db95d167-6480-47ea-8858-3a8c2b0989e4 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:43.281633 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db95d167-6480-47ea-8858-3a8c2b0989e4 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:43.287263 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5678f49a-ae07-4e2c-a8b0-ec4e502f07ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:43.288309 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5678f49a-ae07-4e2c-a8b0-ec4e502f07ca admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:43.288309 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5678f49a-ae07-4e2c-a8b0-ec4e502f07ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:43.288743 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5678f49a-ae07-4e2c-a8b0-ec4e502f07ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:43.289452 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5678f49a-ae07-4e2c-a8b0-ec4e502f07ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:43.291143 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5678f49a-ae07-4e2c-a8b0-ec4e502f07ca admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:43.306184 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1276/2515] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:42:43 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 02:42:43.307612 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5678f49a-ae07-4e2c-a8b0-ec4e502f07ca admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:43.318572 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5678f49a-ae07-4e2c-a8b0-ec4e502f07ca admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:43.329385 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5233cc-5a79-489f-abc1-11b1588c8db9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:43.329385 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5233cc-5a79-489f-abc1-11b1588c8db9 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:43.329385 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5233cc-5a79-489f-abc1-11b1588c8db9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:43.329651 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5233cc-5a79-489f-abc1-11b1588c8db9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:43.330512 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5233cc-5a79-489f-abc1-11b1588c8db9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:43.334170 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5233cc-5a79-489f-abc1-11b1588c8db9 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:43.353712 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1277/2516] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:42:43 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:42:43.354503 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5233cc-5a79-489f-abc1-11b1588c8db9 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:43.356870 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-8e5233cc-5a79-489f-abc1-11b1588c8db9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:43.360152 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1240/2517] 10.222.0.66 () {60 vars in 1143 bytes} [Mon Jul 18 02:42:42 2022] POST /identity/v3/users => generated 286 bytes in 390 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:42:43.361005 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f987b1b6-0da6-4b9b-ab47-50b204052805 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:43.371313 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f987b1b6-0da6-4b9b-ab47-50b204052805 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:43.372062 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5233cc-5a79-489f-abc1-11b1588c8db9 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:43.376995 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5a3fda-ef6e-4ece-883b-f76fd69577af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:43.377183 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdbd95e-d97d-42e1-96a3-7c8a059a6a1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:43.377974 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdbd95e-d97d-42e1-96a3-7c8a059a6a1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:43.378289 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdbd95e-d97d-42e1-96a3-7c8a059a6a1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:43.378568 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdbd95e-d97d-42e1-96a3-7c8a059a6a1c admin admin] PATH_INFO: `/v3/projects/1f36d409ce8e41c09c65c8b244ddbba9` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:43.379949 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdbd95e-d97d-42e1-96a3-7c8a059a6a1c admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f36d409ce8e41c09c65c8b244ddbba9)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:43.380892 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5a3fda-ef6e-4ece-883b-f76fd69577af admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:43.381645 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5a3fda-ef6e-4ece-883b-f76fd69577af admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:43.381881 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdbd95e-d97d-42e1-96a3-7c8a059a6a1c admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:43.382541 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5a3fda-ef6e-4ece-883b-f76fd69577af admin admin] PATH_INFO: `/v3/projects/fb9b0a66f65c4d42ad835d65ea5208e2/users/608f32f746dc4bdd873d9c182a5d6488/roles/65feabf3b27944a8b01a86c8ac608457` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:43.402495 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5a3fda-ef6e-4ece-883b-f76fd69577af admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb9b0a66f65c4d42ad835d65ea5208e2, user_id=608f32f746dc4bdd873d9c182a5d6488, role_id=65feabf3b27944a8b01a86c8ac608457)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:43.405394 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5a3fda-ef6e-4ece-883b-f76fd69577af admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:43.491138 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1241/2518] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:42:43 2022] PUT /identity/v3/projects/fb9b0a66f65c4d42ad835d65ea5208e2/users/608f32f746dc4bdd873d9c182a5d6488/roles/65feabf3b27944a8b01a86c8ac608457 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:42:43.492450 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5a3fda-ef6e-4ece-883b-f76fd69577af admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:43.528660 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-5bdbd95e-d97d-42e1-96a3-7c8a059a6a1c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f36d409ce8e41c09c65c8b244ddbba9', 'request_id': 'req-5bdbd95e-d97d-42e1-96a3-7c8a059a6a1c'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:43.534997 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-5bdbd95e-d97d-42e1-96a3-7c8a059a6a1c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f36d409ce8e41c09c65c8b244ddbba9', 'request_id': 'req-5bdbd95e-d97d-42e1-96a3-7c8a059a6a1c'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:43.567006 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5a3fda-ef6e-4ece-883b-f76fd69577af admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:43.622106 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-5bdbd95e-d97d-42e1-96a3-7c8a059a6a1c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f36d409ce8e41c09c65c8b244ddbba9', 'request_id': 'req-5bdbd95e-d97d-42e1-96a3-7c8a059a6a1c'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:43.634760 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-5bdbd95e-d97d-42e1-96a3-7c8a059a6a1c admin admin] The token cache is being invalidate because project 1f36d409ce8e41c09c65c8b244ddbba9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 02:42:43.635169 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-5bdbd95e-d97d-42e1-96a3-7c8a059a6a1c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:43.636118 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-5bdbd95e-d97d-42e1-96a3-7c8a059a6a1c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:43.639128 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1278/2519] 10.222.0.66 () {58 vars in 1232 bytes} [Mon Jul 18 02:42:43 2022] DELETE /identity/v3/projects/1f36d409ce8e41c09c65c8b244ddbba9 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:42:43.640886 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9ae9ad-729f-470b-9eaf-55487fc3267e None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:43.641293 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9ae9ad-729f-470b-9eaf-55487fc3267e None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:43.641765 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9ae9ad-729f-470b-9eaf-55487fc3267e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:43.643380 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9095df-b6ee-434b-9ef1-ee0811e5e486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:43.644228 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9095df-b6ee-434b-9ef1-ee0811e5e486 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:43.644497 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9095df-b6ee-434b-9ef1-ee0811e5e486 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:43.644849 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9095df-b6ee-434b-9ef1-ee0811e5e486 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:43.645682 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9095df-b6ee-434b-9ef1-ee0811e5e486 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:43.647619 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9095df-b6ee-434b-9ef1-ee0811e5e486 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:43.661428 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1242/2520] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:42:43 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:42:43.662267 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9095df-b6ee-434b-9ef1-ee0811e5e486 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:43.664903 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-7e9095df-b6ee-434b-9ef1-ee0811e5e486 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:43.672367 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-df9ae9ad-729f-470b-9eaf-55487fc3267e None None] MFA Rules not processed for user `608f32f746dc4bdd873d9c182a5d6488`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:42:43.676700 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-df9ae9ad-729f-470b-9eaf-55487fc3267e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:43.740871 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9095df-b6ee-434b-9ef1-ee0811e5e486 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:43.742803 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1279/2521] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:42:43 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:42:43.747170 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9ae9ad-729f-470b-9eaf-55487fc3267e None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:43.748542 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-df9ae9ad-729f-470b-9eaf-55487fc3267e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:43.753841 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a989bf-9b8a-4151-8bbf-08dd14876137 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88936b31709b429f8b6777c673ed7b8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:43.754487 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a989bf-9b8a-4151-8bbf-08dd14876137 service placement] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:43.754844 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a989bf-9b8a-4151-8bbf-08dd14876137 service placement] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:43.755123 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a989bf-9b8a-4151-8bbf-08dd14876137 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:43.757474 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-79a989bf-9b8a-4151-8bbf-08dd14876137 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:43.773801 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a989bf-9b8a-4151-8bbf-08dd14876137 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:43.775551 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a989bf-9b8a-4151-8bbf-08dd14876137 service placement] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:43.777562 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9ae9ad-729f-470b-9eaf-55487fc3267e None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:43.784345 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3808a264-f04c-4360-8435-d2d99e745628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:43.786975 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3808a264-f04c-4360-8435-d2d99e745628 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:43.787521 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3808a264-f04c-4360-8435-d2d99e745628 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:43.788361 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3808a264-f04c-4360-8435-d2d99e745628 admin admin] PATH_INFO: `/v3/users/7e55c6386a794005989a9abbb75e2b4a` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:43.790054 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3808a264-f04c-4360-8435-d2d99e745628 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e55c6386a794005989a9abbb75e2b4a)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:43.790605 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1243/2522] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:42:43 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:42:43.791488 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a989bf-9b8a-4151-8bbf-08dd14876137 service placement] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:43.793157 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-79a989bf-9b8a-4151-8bbf-08dd14876137 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:43.793810 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3808a264-f04c-4360-8435-d2d99e745628 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:43.807433 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a989bf-9b8a-4151-8bbf-08dd14876137 service placement] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:43.814293 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9decd8bd-eece-46fd-9e39-469fa6849319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:43.814978 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9decd8bd-eece-46fd-9e39-469fa6849319 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:43.815460 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9decd8bd-eece-46fd-9e39-469fa6849319 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:43.815920 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9decd8bd-eece-46fd-9e39-469fa6849319 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:43.816841 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9decd8bd-eece-46fd-9e39-469fa6849319 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:43.818898 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9decd8bd-eece-46fd-9e39-469fa6849319 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:43.831384 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1244/2523] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:42:43 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:42:43.832251 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9decd8bd-eece-46fd-9e39-469fa6849319 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:43.836062 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-9decd8bd-eece-46fd-9e39-469fa6849319 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:43.850175 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9decd8bd-eece-46fd-9e39-469fa6849319 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:43.856482 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f59e40a-4294-41e1-aa3e-d9a2ebbd8f25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:43.857173 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f59e40a-4294-41e1-aa3e-d9a2ebbd8f25 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:43.857464 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f59e40a-4294-41e1-aa3e-d9a2ebbd8f25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:43.857746 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f59e40a-4294-41e1-aa3e-d9a2ebbd8f25 admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:43.858493 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f59e40a-4294-41e1-aa3e-d9a2ebbd8f25 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:43.860488 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f59e40a-4294-41e1-aa3e-d9a2ebbd8f25 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:43.971282 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1245/2524] 10.222.0.66 () {60 vars in 1152 bytes} [Mon Jul 18 02:42:43 2022] POST /identity/v3/projects => generated 365 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:42:43.971282 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f59e40a-4294-41e1-aa3e-d9a2ebbd8f25 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:44.025870 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-3808a264-f04c-4360-8435-d2d99e745628 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e55c6386a794005989a9abbb75e2b4a'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:44.038116 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f59e40a-4294-41e1-aa3e-d9a2ebbd8f25 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:44.043322 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66394f9-9c95-4808-ad89-8a06336c116f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:44.044203 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66394f9-9c95-4808-ad89-8a06336c116f admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:44.044474 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66394f9-9c95-4808-ad89-8a06336c116f admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:44.044746 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66394f9-9c95-4808-ad89-8a06336c116f admin admin] PATH_INFO: `/v3/projects/3381cac91ca04a49b37669d4e6dd44b5/users/ccef783c03c94d5c99415b150f3245b8/roles/65feabf3b27944a8b01a86c8ac608457` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:44.059482 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66394f9-9c95-4808-ad89-8a06336c116f admin admin] RBAC: Authorizing `identity:create_grant(project_id=3381cac91ca04a49b37669d4e6dd44b5, user_id=ccef783c03c94d5c99415b150f3245b8, role_id=65feabf3b27944a8b01a86c8ac608457)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:44.067241 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66394f9-9c95-4808-ad89-8a06336c116f admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:44.085860 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-3808a264-f04c-4360-8435-d2d99e745628 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e55c6386a794005989a9abbb75e2b4a'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:44.093227 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-3808a264-f04c-4360-8435-d2d99e745628 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e55c6386a794005989a9abbb75e2b4a'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:44.093945 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-3808a264-f04c-4360-8435-d2d99e745628 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e55c6386a794005989a9abbb75e2b4a'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:44.106289 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-3808a264-f04c-4360-8435-d2d99e745628 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e55c6386a794005989a9abbb75e2b4a'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:44.112289 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1280/2525] 10.222.0.66 () {58 vars in 1223 bytes} [Mon Jul 18 02:42:43 2022] DELETE /identity/v3/users/7e55c6386a794005989a9abbb75e2b4a => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:42:44.113055 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3808a264-f04c-4360-8435-d2d99e745628 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:44.114503 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-3808a264-f04c-4360-8435-d2d99e745628 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:44.174625 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1246/2526] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:42:43 2022] PUT /identity/v3/projects/3381cac91ca04a49b37669d4e6dd44b5/users/ccef783c03c94d5c99415b150f3245b8/roles/65feabf3b27944a8b01a86c8ac608457 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:42:44.175714 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66394f9-9c95-4808-ad89-8a06336c116f admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:44.181884 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3808a264-f04c-4360-8435-d2d99e745628 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:44.186683 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-c66394f9-9c95-4808-ad89-8a06336c116f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:44.241272 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cde5d2-0d3a-4a49-a2cc-e1d6c3a40098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:44.242244 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cde5d2-0d3a-4a49-a2cc-e1d6c3a40098 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:44.242525 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cde5d2-0d3a-4a49-a2cc-e1d6c3a40098 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:44.242799 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cde5d2-0d3a-4a49-a2cc-e1d6c3a40098 admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:44.243487 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cde5d2-0d3a-4a49-a2cc-e1d6c3a40098 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:44.246072 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cde5d2-0d3a-4a49-a2cc-e1d6c3a40098 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:44.259190 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66394f9-9c95-4808-ad89-8a06336c116f admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:44.264713 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22d857b-1860-49d9-ac00-06bfecdb857a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:44.265587 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22d857b-1860-49d9-ac00-06bfecdb857a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:44.265900 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22d857b-1860-49d9-ac00-06bfecdb857a admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:44.266197 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22d857b-1860-49d9-ac00-06bfecdb857a admin admin] PATH_INFO: `/v3/users/7ddb197852db4b20a84aa6c0862468ef` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:44.267430 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22d857b-1860-49d9-ac00-06bfecdb857a admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ddb197852db4b20a84aa6c0862468ef)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:44.269326 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22d857b-1860-49d9-ac00-06bfecdb857a admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:44.281096 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:42:44.352823 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1281/2527] 10.222.0.66 () {60 vars in 1143 bytes} [Mon Jul 18 02:42:44 2022] POST /identity/v3/users => generated 289 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:42:44.352823 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00350648-828a-484b-b959-c9dbd443a615 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:44.352823 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00350648-828a-484b-b959-c9dbd443a615 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:44.352823 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00350648-828a-484b-b959-c9dbd443a615 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:44.381229 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-00350648-828a-484b-b959-c9dbd443a615 None None] MFA Rules not processed for user `ccef783c03c94d5c99415b150f3245b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:42:44.382538 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-00350648-828a-484b-b959-c9dbd443a615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:44.449184 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1282/2528] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:42:44 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:42:44.449753 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00350648-828a-484b-b959-c9dbd443a615 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:44.459460 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:42:44.462272 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00350648-828a-484b-b959-c9dbd443a615 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:44.467627 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19211002-e6e7-4f5c-a006-a51070c577e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:44.469164 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19211002-e6e7-4f5c-a006-a51070c577e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:44.469164 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19211002-e6e7-4f5c-a006-a51070c577e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:44.469164 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19211002-e6e7-4f5c-a006-a51070c577e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:44.469554 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19211002-e6e7-4f5c-a006-a51070c577e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:44.471190 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19211002-e6e7-4f5c-a006-a51070c577e3 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:44.482045 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1283/2529] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:42:44 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:42:44.482946 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19211002-e6e7-4f5c-a006-a51070c577e3 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:44.484586 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-19211002-e6e7-4f5c-a006-a51070c577e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:44.500316 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-a22d857b-1860-49d9-ac00-06bfecdb857a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ddb197852db4b20a84aa6c0862468ef'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:44.541526 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19211002-e6e7-4f5c-a006-a51070c577e3 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:44.545435 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f944479-2c13-44fd-800d-9759ac4e9148 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd49388facdf446885e55b45ef91f01e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:44.546154 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f944479-2c13-44fd-800d-9759ac4e9148 service cinder] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:44.546423 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f944479-2c13-44fd-800d-9759ac4e9148 service cinder] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:44.546697 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f944479-2c13-44fd-800d-9759ac4e9148 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:44.555136 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f944479-2c13-44fd-800d-9759ac4e9148 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:44.558999 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f944479-2c13-44fd-800d-9759ac4e9148 service cinder] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:44.566340 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1284/2530] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:42:44 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:42:44.567183 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f944479-2c13-44fd-800d-9759ac4e9148 service cinder] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:44.579629 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f944479-2c13-44fd-800d-9759ac4e9148 service cinder] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:44.584631 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8a679d-3589-4c3e-b8e4-aea730c33efe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:44.585608 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8a679d-3589-4c3e-b8e4-aea730c33efe admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:44.585893 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8a679d-3589-4c3e-b8e4-aea730c33efe admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:44.586223 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8a679d-3589-4c3e-b8e4-aea730c33efe admin admin] PATH_INFO: `/v3/projects/4090f7541d8042f4bdd8e2af4bc1f52d/users/4bded16043ca4b7cb1c96bdbb94a741d/roles/d4aa3324f0c84887b41d0dc4acf7f30e` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:44.595010 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-a22d857b-1860-49d9-ac00-06bfecdb857a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ddb197852db4b20a84aa6c0862468ef'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:44.595668 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-a22d857b-1860-49d9-ac00-06bfecdb857a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ddb197852db4b20a84aa6c0862468ef'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:44.606981 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8a679d-3589-4c3e-b8e4-aea730c33efe admin admin] RBAC: Authorizing `identity:create_grant(project_id=4090f7541d8042f4bdd8e2af4bc1f52d, user_id=4bded16043ca4b7cb1c96bdbb94a741d, role_id=d4aa3324f0c84887b41d0dc4acf7f30e)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:44.609447 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8a679d-3589-4c3e-b8e4-aea730c33efe admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:44.609888 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-a22d857b-1860-49d9-ac00-06bfecdb857a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ddb197852db4b20a84aa6c0862468ef'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:44.610652 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-a22d857b-1860-49d9-ac00-06bfecdb857a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ddb197852db4b20a84aa6c0862468ef'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:44.625591 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1247/2531] 10.222.0.66 () {58 vars in 1223 bytes} [Mon Jul 18 02:42:44 2022] DELETE /identity/v3/users/7ddb197852db4b20a84aa6c0862468ef => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:42:44.629874 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22d857b-1860-49d9-ac00-06bfecdb857a admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:44.629874 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-a22d857b-1860-49d9-ac00-06bfecdb857a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:44.676362 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1285/2532] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:42:44 2022] PUT /identity/v3/projects/4090f7541d8042f4bdd8e2af4bc1f52d/users/4bded16043ca4b7cb1c96bdbb94a741d/roles/d4aa3324f0c84887b41d0dc4acf7f30e => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:42:44.677158 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8a679d-3589-4c3e-b8e4-aea730c33efe admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:44.678802 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-4d8a679d-3589-4c3e-b8e4-aea730c33efe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:44.694935 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22d857b-1860-49d9-ac00-06bfecdb857a admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:44.750141 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8a679d-3589-4c3e-b8e4-aea730c33efe admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:44.755169 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a265e4d3-04ff-46d9-b75f-78cc2a865030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:44.756072 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a265e4d3-04ff-46d9-b75f-78cc2a865030 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:44.756812 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a265e4d3-04ff-46d9-b75f-78cc2a865030 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:44.757097 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a265e4d3-04ff-46d9-b75f-78cc2a865030 admin admin] PATH_INFO: `/v3/users/ccef783c03c94d5c99415b150f3245b8` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:44.757831 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30d0458-5085-4dcf-856d-a4dfb7d1210e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:44.758736 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30d0458-5085-4dcf-856d-a4dfb7d1210e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:44.758846 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a265e4d3-04ff-46d9-b75f-78cc2a865030 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ccef783c03c94d5c99415b150f3245b8)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:44.759111 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30d0458-5085-4dcf-856d-a4dfb7d1210e admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:44.759395 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30d0458-5085-4dcf-856d-a4dfb7d1210e admin admin] PATH_INFO: `/v3/projects/b4150e808c9443c8ac7652a31b5cce5e` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:44.760597 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30d0458-5085-4dcf-856d-a4dfb7d1210e admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4150e808c9443c8ac7652a31b5cce5e)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:44.760791 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a265e4d3-04ff-46d9-b75f-78cc2a865030 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:44.762698 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30d0458-5085-4dcf-856d-a4dfb7d1210e admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:44.857805 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:42:44.926864 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-e30d0458-5085-4dcf-856d-a4dfb7d1210e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4150e808c9443c8ac7652a31b5cce5e', 'request_id': 'req-e30d0458-5085-4dcf-856d-a4dfb7d1210e'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:44.927403 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-a265e4d3-04ff-46d9-b75f-78cc2a865030 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ccef783c03c94d5c99415b150f3245b8'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:44.932435 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-e30d0458-5085-4dcf-856d-a4dfb7d1210e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4150e808c9443c8ac7652a31b5cce5e', 'request_id': 'req-e30d0458-5085-4dcf-856d-a4dfb7d1210e'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:44.941283 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-e30d0458-5085-4dcf-856d-a4dfb7d1210e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4150e808c9443c8ac7652a31b5cce5e', 'request_id': 'req-e30d0458-5085-4dcf-856d-a4dfb7d1210e'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:45.017592 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-a265e4d3-04ff-46d9-b75f-78cc2a865030 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ccef783c03c94d5c99415b150f3245b8'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:45.024606 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-a265e4d3-04ff-46d9-b75f-78cc2a865030 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ccef783c03c94d5c99415b150f3245b8'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:45.025286 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-a265e4d3-04ff-46d9-b75f-78cc2a865030 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ccef783c03c94d5c99415b150f3245b8'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:45.037164 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-a265e4d3-04ff-46d9-b75f-78cc2a865030 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ccef783c03c94d5c99415b150f3245b8'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:45.043618 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1286/2533] 10.222.0.66 () {58 vars in 1223 bytes} [Mon Jul 18 02:42:44 2022] DELETE /identity/v3/users/ccef783c03c94d5c99415b150f3245b8 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:42:45.044470 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a265e4d3-04ff-46d9-b75f-78cc2a865030 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:45.045819 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-a265e4d3-04ff-46d9-b75f-78cc2a865030 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:45.085106 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-e30d0458-5085-4dcf-856d-a4dfb7d1210e admin admin] The token cache is being invalidate because project b4150e808c9443c8ac7652a31b5cce5e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 02:42:45.085401 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-e30d0458-5085-4dcf-856d-a4dfb7d1210e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:45.086106 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-e30d0458-5085-4dcf-856d-a4dfb7d1210e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:45.087686 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1248/2534] 10.222.0.66 () {58 vars in 1232 bytes} [Mon Jul 18 02:42:44 2022] DELETE /identity/v3/projects/b4150e808c9443c8ac7652a31b5cce5e => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:42:45.088529 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30d0458-5085-4dcf-856d-a4dfb7d1210e admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:45.090726 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-e30d0458-5085-4dcf-856d-a4dfb7d1210e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:45.114389 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a265e4d3-04ff-46d9-b75f-78cc2a865030 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:45.119079 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43129e7-2d9f-4ca4-b6e1-9a83fc646d97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:45.119739 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43129e7-2d9f-4ca4-b6e1-9a83fc646d97 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:45.120025 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43129e7-2d9f-4ca4-b6e1-9a83fc646d97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:45.120391 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43129e7-2d9f-4ca4-b6e1-9a83fc646d97 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:45.121237 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43129e7-2d9f-4ca4-b6e1-9a83fc646d97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:45.122881 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43129e7-2d9f-4ca4-b6e1-9a83fc646d97 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:45.132842 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1287/2535] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:42:45 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:42:45.134176 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8dc205-daed-4943-8998-bf7eb2e40104 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:45.134477 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8dc205-daed-4943-8998-bf7eb2e40104 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:45.134745 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8dc205-daed-4943-8998-bf7eb2e40104 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:45.152932 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-2b8dc205-daed-4943-8998-bf7eb2e40104 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:42:45.154167 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-2b8dc205-daed-4943-8998-bf7eb2e40104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:45.158149 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30d0458-5085-4dcf-856d-a4dfb7d1210e admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:45.161733 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1288/2536] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:42:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:42:45.162530 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8dc205-daed-4943-8998-bf7eb2e40104 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:45.162908 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8a9b66-060f-44f9-913d-dc53b0506fc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:45.163777 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8a9b66-060f-44f9-913d-dc53b0506fc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:45.164110 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8a9b66-060f-44f9-913d-dc53b0506fc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:45.164393 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8a9b66-060f-44f9-913d-dc53b0506fc5 admin admin] PATH_INFO: `/v3/projects/f3c1745729fe47238223557221c2e905` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:45.165603 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8a9b66-060f-44f9-913d-dc53b0506fc5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3c1745729fe47238223557221c2e905)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:45.165869 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-2b8dc205-daed-4943-8998-bf7eb2e40104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:45.168451 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8a9b66-060f-44f9-913d-dc53b0506fc5 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:45.177990 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:42:45.178714 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8dc205-daed-4943-8998-bf7eb2e40104 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:45.186633 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebbe6fb-2a14-46fb-bbce-30835eddb2e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:45.186633 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebbe6fb-2a14-46fb-bbce-30835eddb2e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:45.186633 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebbe6fb-2a14-46fb-bbce-30835eddb2e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:45.186633 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebbe6fb-2a14-46fb-bbce-30835eddb2e7 admin admin] PATH_INFO: `/v3/users/cb4fbc5b4f814e10b9c61968e4fbc253` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:45.187270 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebbe6fb-2a14-46fb-bbce-30835eddb2e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb4fbc5b4f814e10b9c61968e4fbc253)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:45.189027 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebbe6fb-2a14-46fb-bbce-30835eddb2e7 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:45.303831 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-9c8a9b66-060f-44f9-913d-dc53b0506fc5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3c1745729fe47238223557221c2e905', 'request_id': 'req-9c8a9b66-060f-44f9-913d-dc53b0506fc5'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:45.309341 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-9c8a9b66-060f-44f9-913d-dc53b0506fc5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3c1745729fe47238223557221c2e905', 'request_id': 'req-9c8a9b66-060f-44f9-913d-dc53b0506fc5'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:45.315693 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-9c8a9b66-060f-44f9-913d-dc53b0506fc5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3c1745729fe47238223557221c2e905', 'request_id': 'req-9c8a9b66-060f-44f9-913d-dc53b0506fc5'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:45.457135 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-9c8a9b66-060f-44f9-913d-dc53b0506fc5 admin admin] The token cache is being invalidate because project f3c1745729fe47238223557221c2e905 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 02:42:45.457424 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-9c8a9b66-060f-44f9-913d-dc53b0506fc5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:45.458077 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-9c8a9b66-060f-44f9-913d-dc53b0506fc5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:45.459690 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1249/2537] 10.222.0.66 () {58 vars in 1232 bytes} [Mon Jul 18 02:42:45 2022] DELETE /identity/v3/projects/f3c1745729fe47238223557221c2e905 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:42:45.460450 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8a9b66-060f-44f9-913d-dc53b0506fc5 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:45.461990 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-9c8a9b66-060f-44f9-913d-dc53b0506fc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:45.466408 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-eebbe6fb-2a14-46fb-bbce-30835eddb2e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb4fbc5b4f814e10b9c61968e4fbc253'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:45.525762 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8a9b66-060f-44f9-913d-dc53b0506fc5 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:45.531143 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50b7625-dc79-4872-88d7-8a8579d9dbaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:45.532036 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50b7625-dc79-4872-88d7-8a8579d9dbaf admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:45.532331 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50b7625-dc79-4872-88d7-8a8579d9dbaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:45.532604 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50b7625-dc79-4872-88d7-8a8579d9dbaf admin admin] PATH_INFO: `/v3/projects/4090f7541d8042f4bdd8e2af4bc1f52d/users/4bded16043ca4b7cb1c96bdbb94a741d/roles/d4aa3324f0c84887b41d0dc4acf7f30e` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:45.534780 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50b7625-dc79-4872-88d7-8a8579d9dbaf admin admin] RBAC: Authorizing `identity:create_grant(project_id=4090f7541d8042f4bdd8e2af4bc1f52d, user_id=4bded16043ca4b7cb1c96bdbb94a741d, role_id=d4aa3324f0c84887b41d0dc4acf7f30e)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:45.537855 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50b7625-dc79-4872-88d7-8a8579d9dbaf admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:45.548704 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-eebbe6fb-2a14-46fb-bbce-30835eddb2e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb4fbc5b4f814e10b9c61968e4fbc253'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:45.549258 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1250/2538] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:42:45 2022] PUT /identity/v3/projects/4090f7541d8042f4bdd8e2af4bc1f52d/users/4bded16043ca4b7cb1c96bdbb94a741d/roles/d4aa3324f0c84887b41d0dc4acf7f30e => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:42:45.549857 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50b7625-dc79-4872-88d7-8a8579d9dbaf admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:45.551399 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-a50b7625-dc79-4872-88d7-8a8579d9dbaf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:45.557176 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-eebbe6fb-2a14-46fb-bbce-30835eddb2e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb4fbc5b4f814e10b9c61968e4fbc253'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:45.559538 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-eebbe6fb-2a14-46fb-bbce-30835eddb2e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb4fbc5b4f814e10b9c61968e4fbc253'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:45.571065 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-eebbe6fb-2a14-46fb-bbce-30835eddb2e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb4fbc5b4f814e10b9c61968e4fbc253'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:45.577583 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1289/2539] 10.222.0.66 () {58 vars in 1223 bytes} [Mon Jul 18 02:42:45 2022] DELETE /identity/v3/users/cb4fbc5b4f814e10b9c61968e4fbc253 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:42:45.579990 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c917fe6c-9817-4323-aa1a-3e682fc18352 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:45.579990 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c917fe6c-9817-4323-aa1a-3e682fc18352 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:45.579990 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c917fe6c-9817-4323-aa1a-3e682fc18352 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:45.604071 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-c917fe6c-9817-4323-aa1a-3e682fc18352 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:42:45.605587 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-c917fe6c-9817-4323-aa1a-3e682fc18352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:45.617116 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50b7625-dc79-4872-88d7-8a8579d9dbaf admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:45.671433 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163a7693-90f5-4a03-b0d1-662377b26949 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eaf19a259c2741cea375a38763e33cdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:45.672090 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163a7693-90f5-4a03-b0d1-662377b26949 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:45.672380 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163a7693-90f5-4a03-b0d1-662377b26949 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:45.672664 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163a7693-90f5-4a03-b0d1-662377b26949 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:45.674518 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-163a7693-90f5-4a03-b0d1-662377b26949 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:45.678456 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1290/2540] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:42:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:42:45.680124 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc7efa7-5d7e-4f33-9485-c1e7fb164eef None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:45.680832 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc7efa7-5d7e-4f33-9485-c1e7fb164eef None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:45.680832 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc7efa7-5d7e-4f33-9485-c1e7fb164eef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:45.693967 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163a7693-90f5-4a03-b0d1-662377b26949 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:45.695492 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163a7693-90f5-4a03-b0d1-662377b26949 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:45.706212 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1251/2541] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:42:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:42:45.707406 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a9787d-eccd-4723-8e34-77b0a15a7e3b None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:45.707651 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a9787d-eccd-4723-8e34-77b0a15a7e3b None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:45.707909 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a9787d-eccd-4723-8e34-77b0a15a7e3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:45.712689 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-5dc7efa7-5d7e-4f33-9485-c1e7fb164eef None None] MFA Rules not processed for user `4bded16043ca4b7cb1c96bdbb94a741d`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:42:45.714060 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-5dc7efa7-5d7e-4f33-9485-c1e7fb164eef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:45.727420 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-a7a9787d-eccd-4723-8e34-77b0a15a7e3b None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:42:45.728968 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-a7a9787d-eccd-4723-8e34-77b0a15a7e3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:45.738388 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1252/2542] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:42:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:42:45.755248 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a9787d-eccd-4723-8e34-77b0a15a7e3b None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:45.765304 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a9787d-eccd-4723-8e34-77b0a15a7e3b None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:45.770807 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46c71af-c38c-4061-9255-35cd0a13e905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:45.771465 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46c71af-c38c-4061-9255-35cd0a13e905 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:45.771764 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46c71af-c38c-4061-9255-35cd0a13e905 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:45.772014 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46c71af-c38c-4061-9255-35cd0a13e905 admin admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:45.772730 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46c71af-c38c-4061-9255-35cd0a13e905 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:42:45.773033 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46c71af-c38c-4061-9255-35cd0a13e905 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:45.774797 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46c71af-c38c-4061-9255-35cd0a13e905 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:45.787773 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1291/2543] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:42:45 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:42:45.789121 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1253/2544] 10.222.0.66 () {58 vars in 1152 bytes} [Mon Jul 18 02:42:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:42:45.793926 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc7efa7-5d7e-4f33-9485-c1e7fb164eef None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:45.800021 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46c71af-c38c-4061-9255-35cd0a13e905 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:45.801834 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-a46c71af-c38c-4061-9255-35cd0a13e905 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:45.805069 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc7efa7-5d7e-4f33-9485-c1e7fb164eef None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:45.809889 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5891dd-9752-4334-94d6-44aaaf29d6a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:45.810733 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5891dd-9752-4334-94d6-44aaaf29d6a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:45.810956 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5891dd-9752-4334-94d6-44aaaf29d6a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:45.811239 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5891dd-9752-4334-94d6-44aaaf29d6a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:45.812539 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5891dd-9752-4334-94d6-44aaaf29d6a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:45.814191 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5891dd-9752-4334-94d6-44aaaf29d6a3 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:45.868056 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46c71af-c38c-4061-9255-35cd0a13e905 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:45.873187 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c1ea2d-20e3-49e6-8659-3c0d3651a64c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd49388facdf446885e55b45ef91f01e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:45.873851 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c1ea2d-20e3-49e6-8659-3c0d3651a64c service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:45.874190 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c1ea2d-20e3-49e6-8659-3c0d3651a64c service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:45.874471 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c1ea2d-20e3-49e6-8659-3c0d3651a64c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:45.884319 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c1ea2d-20e3-49e6-8659-3c0d3651a64c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:45.885845 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c1ea2d-20e3-49e6-8659-3c0d3651a64c service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:45.896751 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1254/2545] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:42:45 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 02:42:46.010563 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1292/2546] 10.222.0.66 () {60 vars in 1152 bytes} [Mon Jul 18 02:42:45 2022] POST /identity/v3/projects => generated 367 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 02:42:46.012546 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c1ea2d-20e3-49e6-8659-3c0d3651a64c service cinder] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:46.023152 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c1ea2d-20e3-49e6-8659-3c0d3651a64c service cinder] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:46.084210 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaf3677-d02b-40f0-b7ca-e93555429015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:46.084898 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaf3677-d02b-40f0-b7ca-e93555429015 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:46.085182 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaf3677-d02b-40f0-b7ca-e93555429015 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:46.085462 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaf3677-d02b-40f0-b7ca-e93555429015 admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:46.086180 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaf3677-d02b-40f0-b7ca-e93555429015 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:46.088119 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaf3677-d02b-40f0-b7ca-e93555429015 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:46.106854 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5891dd-9752-4334-94d6-44aaaf29d6a3 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:46.108573 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-ce5891dd-9752-4334-94d6-44aaaf29d6a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:46.122779 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5891dd-9752-4334-94d6-44aaaf29d6a3 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:46.128393 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6ddc39-617a-433e-b1f3-b3e48d2e6038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:46.129402 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6ddc39-617a-433e-b1f3-b3e48d2e6038 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:46.129794 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6ddc39-617a-433e-b1f3-b3e48d2e6038 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:46.131583 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6ddc39-617a-433e-b1f3-b3e48d2e6038 admin admin] PATH_INFO: `/v3/projects/2815332cd1d3486291843a8af1aa95ee` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:46.133414 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6ddc39-617a-433e-b1f3-b3e48d2e6038 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2815332cd1d3486291843a8af1aa95ee)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:46.140376 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6ddc39-617a-433e-b1f3-b3e48d2e6038 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:46.169381 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:42:46.206136 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1255/2547] 10.222.0.66 () {60 vars in 1143 bytes} [Mon Jul 18 02:42:46 2022] POST /identity/v3/users => generated 284 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:42:46.207509 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaf3677-d02b-40f0-b7ca-e93555429015 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:46.220487 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaf3677-d02b-40f0-b7ca-e93555429015 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:46.225609 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d697dd5d-2d18-438f-b198-3b52ad05dd9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:46.226423 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d697dd5d-2d18-438f-b198-3b52ad05dd9b admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:46.231733 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d697dd5d-2d18-438f-b198-3b52ad05dd9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:46.231733 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d697dd5d-2d18-438f-b198-3b52ad05dd9b admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:46.231733 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d697dd5d-2d18-438f-b198-3b52ad05dd9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:46.231733 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d697dd5d-2d18-438f-b198-3b52ad05dd9b admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:46.241316 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1256/2548] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:42:46 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 02:42:46.242542 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d697dd5d-2d18-438f-b198-3b52ad05dd9b admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:46.306068 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d697dd5d-2d18-438f-b198-3b52ad05dd9b admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:46.309941 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:42:46.325846 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-df6ddc39-617a-433e-b1f3-b3e48d2e6038 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2815332cd1d3486291843a8af1aa95ee', 'request_id': 'req-df6ddc39-617a-433e-b1f3-b3e48d2e6038'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:46.343373 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-df6ddc39-617a-433e-b1f3-b3e48d2e6038 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2815332cd1d3486291843a8af1aa95ee', 'request_id': 'req-df6ddc39-617a-433e-b1f3-b3e48d2e6038'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:46.367415 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2a86d6-477a-4c32-9e1f-68743cce072b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd49388facdf446885e55b45ef91f01e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:46.368228 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2a86d6-477a-4c32-9e1f-68743cce072b service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:46.368609 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2a86d6-477a-4c32-9e1f-68743cce072b service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:46.369090 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2a86d6-477a-4c32-9e1f-68743cce072b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:46.371031 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-5e2a86d6-477a-4c32-9e1f-68743cce072b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:46.428685 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-df6ddc39-617a-433e-b1f3-b3e48d2e6038 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2815332cd1d3486291843a8af1aa95ee', 'request_id': 'req-df6ddc39-617a-433e-b1f3-b3e48d2e6038'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:46.433333 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2a86d6-477a-4c32-9e1f-68743cce072b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:46.434924 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2a86d6-477a-4c32-9e1f-68743cce072b service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:46.441324 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-df6ddc39-617a-433e-b1f3-b3e48d2e6038 admin admin] The token cache is being invalidate because project 2815332cd1d3486291843a8af1aa95ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 02:42:46.441612 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-df6ddc39-617a-433e-b1f3-b3e48d2e6038 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:46.442312 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-df6ddc39-617a-433e-b1f3-b3e48d2e6038 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:46.443910 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1293/2549] 10.222.0.66 () {58 vars in 1232 bytes} [Mon Jul 18 02:42:46 2022] DELETE /identity/v3/projects/2815332cd1d3486291843a8af1aa95ee => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 02:42:46.444719 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6ddc39-617a-433e-b1f3-b3e48d2e6038 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:46.445991 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-df6ddc39-617a-433e-b1f3-b3e48d2e6038 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:46.450072 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1257/2550] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:42:46 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:42:46.450886 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2a86d6-477a-4c32-9e1f-68743cce072b service cinder] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:46.452318 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-5e2a86d6-477a-4c32-9e1f-68743cce072b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:46.517954 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6ddc39-617a-433e-b1f3-b3e48d2e6038 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:46.522310 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2a86d6-477a-4c32-9e1f-68743cce072b service cinder] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:46.523591 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec9ad53-8d41-4414-afd6-5fd02d620af1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:46.524410 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec9ad53-8d41-4414-afd6-5fd02d620af1 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:46.524707 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec9ad53-8d41-4414-afd6-5fd02d620af1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:46.525119 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec9ad53-8d41-4414-afd6-5fd02d620af1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:46.526338 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec9ad53-8d41-4414-afd6-5fd02d620af1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:46.528670 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56602363-9c29-48b7-b3a5-93c7198a3acb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:46.528948 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec9ad53-8d41-4414-afd6-5fd02d620af1 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:46.530064 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56602363-9c29-48b7-b3a5-93c7198a3acb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:46.530531 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56602363-9c29-48b7-b3a5-93c7198a3acb admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:46.530814 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56602363-9c29-48b7-b3a5-93c7198a3acb admin admin] PATH_INFO: `/v3/projects/3381cac91ca04a49b37669d4e6dd44b5` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:46.532226 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56602363-9c29-48b7-b3a5-93c7198a3acb admin admin] RBAC: Authorizing `identity:delete_project(project_id=3381cac91ca04a49b37669d4e6dd44b5)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:46.534371 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56602363-9c29-48b7-b3a5-93c7198a3acb admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:46.541216 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1294/2551] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:42:46 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:42:46.546423 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d2e76b-d2a4-4afe-8537-73e23b66e796 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:46.551064 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d2e76b-d2a4-4afe-8537-73e23b66e796 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:46.551064 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d2e76b-d2a4-4afe-8537-73e23b66e796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:46.568828 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-b9d2e76b-d2a4-4afe-8537-73e23b66e796 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:42:46.571213 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-b9d2e76b-d2a4-4afe-8537-73e23b66e796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:46.580265 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1295/2552] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:42:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:42:46.584919 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d2e76b-d2a4-4afe-8537-73e23b66e796 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:46.591105 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d2e76b-d2a4-4afe-8537-73e23b66e796 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:46.596192 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ebfde1-4cd1-4a47-b369-0c830c8a86b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:46.598327 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ebfde1-4cd1-4a47-b369-0c830c8a86b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:46.598659 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ebfde1-4cd1-4a47-b369-0c830c8a86b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:46.602852 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ebfde1-4cd1-4a47-b369-0c830c8a86b2 admin admin] PATH_INFO: `/v3/projects/670634a51ed94c67beaf0d1b4ffc38e9/users/1b332bd700fc48efa70715c13a0ba6a9/roles/65feabf3b27944a8b01a86c8ac608457` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:46.614828 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ebfde1-4cd1-4a47-b369-0c830c8a86b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=670634a51ed94c67beaf0d1b4ffc38e9, user_id=1b332bd700fc48efa70715c13a0ba6a9, role_id=65feabf3b27944a8b01a86c8ac608457)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:46.617476 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ebfde1-4cd1-4a47-b369-0c830c8a86b2 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:46.730931 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1296/2553] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:42:46 2022] PUT /identity/v3/projects/670634a51ed94c67beaf0d1b4ffc38e9/users/1b332bd700fc48efa70715c13a0ba6a9/roles/65feabf3b27944a8b01a86c8ac608457 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:42:46.732574 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743bf6ff-8723-4367-979f-25c206661bd6 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:46.732871 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743bf6ff-8723-4367-979f-25c206661bd6 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:46.733148 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743bf6ff-8723-4367-979f-25c206661bd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:46.739376 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-56602363-9c29-48b7-b3a5-93c7198a3acb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3381cac91ca04a49b37669d4e6dd44b5', 'request_id': 'req-56602363-9c29-48b7-b3a5-93c7198a3acb'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:46.746817 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-56602363-9c29-48b7-b3a5-93c7198a3acb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3381cac91ca04a49b37669d4e6dd44b5', 'request_id': 'req-56602363-9c29-48b7-b3a5-93c7198a3acb'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:46.753742 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-56602363-9c29-48b7-b3a5-93c7198a3acb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3381cac91ca04a49b37669d4e6dd44b5', 'request_id': 'req-56602363-9c29-48b7-b3a5-93c7198a3acb'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:46.759804 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-743bf6ff-8723-4367-979f-25c206661bd6 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:42:46.761147 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-743bf6ff-8723-4367-979f-25c206661bd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:46.819244 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1297/2554] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:42:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:42:46.820605 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86fbd93-5f9e-4ae0-9ab7-d21dec22d3aa None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:46.820896 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86fbd93-5f9e-4ae0-9ab7-d21dec22d3aa None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:46.821174 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86fbd93-5f9e-4ae0-9ab7-d21dec22d3aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:46.848301 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-e86fbd93-5f9e-4ae0-9ab7-d21dec22d3aa None None] MFA Rules not processed for user `1b332bd700fc48efa70715c13a0ba6a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:42:46.849555 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-e86fbd93-5f9e-4ae0-9ab7-d21dec22d3aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:46.867058 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-56602363-9c29-48b7-b3a5-93c7198a3acb admin admin] The token cache is being invalidate because project 3381cac91ca04a49b37669d4e6dd44b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 02:42:46.867355 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-56602363-9c29-48b7-b3a5-93c7198a3acb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:46.868004 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-56602363-9c29-48b7-b3a5-93c7198a3acb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:46.869577 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1258/2555] 10.222.0.66 () {58 vars in 1232 bytes} [Mon Jul 18 02:42:46 2022] DELETE /identity/v3/projects/3381cac91ca04a49b37669d4e6dd44b5 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:42:46.870379 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56602363-9c29-48b7-b3a5-93c7198a3acb admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:46.872747 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-56602363-9c29-48b7-b3a5-93c7198a3acb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:46.885369 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56602363-9c29-48b7-b3a5-93c7198a3acb admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:46.890978 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9e8fd7-a453-4d1b-8f66-09ecad7f2e33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:46.891635 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9e8fd7-a453-4d1b-8f66-09ecad7f2e33 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:46.891919 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9e8fd7-a453-4d1b-8f66-09ecad7f2e33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:46.892198 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9e8fd7-a453-4d1b-8f66-09ecad7f2e33 admin admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:46.892926 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9e8fd7-a453-4d1b-8f66-09ecad7f2e33 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:42:46.893231 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9e8fd7-a453-4d1b-8f66-09ecad7f2e33 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:46.894998 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9e8fd7-a453-4d1b-8f66-09ecad7f2e33 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:46.906809 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1259/2556] 10.222.0.66 () {58 vars in 1152 bytes} [Mon Jul 18 02:42:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:42:46.908100 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615ef1d1-8130-4be4-9b58-09ad64312ab7 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:46.908386 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615ef1d1-8130-4be4-9b58-09ad64312ab7 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:46.908663 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615ef1d1-8130-4be4-9b58-09ad64312ab7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:46.916521 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1298/2557] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:42:46 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:42:46.917136 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86fbd93-5f9e-4ae0-9ab7-d21dec22d3aa None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:46.928383 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-615ef1d1-8130-4be4-9b58-09ad64312ab7 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:42:46.928591 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86fbd93-5f9e-4ae0-9ab7-d21dec22d3aa None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:46.929721 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-615ef1d1-8130-4be4-9b58-09ad64312ab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:46.933799 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc265967-faad-47f5-aa1f-c920c4d4e557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:46.934499 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc265967-faad-47f5-aa1f-c920c4d4e557 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:46.934787 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc265967-faad-47f5-aa1f-c920c4d4e557 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:46.935065 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc265967-faad-47f5-aa1f-c920c4d4e557 admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:46.935871 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc265967-faad-47f5-aa1f-c920c4d4e557 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:46.938047 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc265967-faad-47f5-aa1f-c920c4d4e557 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:46.939577 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1260/2558] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:42:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:42:46.946602 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615ef1d1-8130-4be4-9b58-09ad64312ab7 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:46.949270 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-615ef1d1-8130-4be4-9b58-09ad64312ab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:46.950879 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:42:46.961094 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615ef1d1-8130-4be4-9b58-09ad64312ab7 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:46.966028 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15908c59-9e02-405f-9bd8-88454521d9ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:46.966769 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15908c59-9e02-405f-9bd8-88454521d9ab admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:46.966984 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15908c59-9e02-405f-9bd8-88454521d9ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:46.967253 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15908c59-9e02-405f-9bd8-88454521d9ab admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:46.967938 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15908c59-9e02-405f-9bd8-88454521d9ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:46.969848 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15908c59-9e02-405f-9bd8-88454521d9ab admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:47.039192 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:42:47.077861 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1299/2559] 10.222.0.66 () {60 vars in 1152 bytes} [Mon Jul 18 02:42:46 2022] POST /identity/v3/projects => generated 373 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:42:47.078043 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1261/2560] 10.222.0.66 () {60 vars in 1152 bytes} [Mon Jul 18 02:42:46 2022] POST /identity/v3/projects => generated 367 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:42:47.078386 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9dc9c6-1bff-42e4-8438-7d67a8cf0169 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:47.078386 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9dc9c6-1bff-42e4-8438-7d67a8cf0169 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:47.078386 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9dc9c6-1bff-42e4-8438-7d67a8cf0169 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:47.079151 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15908c59-9e02-405f-9bd8-88454521d9ab admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:47.095711 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-dc9dc9c6-1bff-42e4-8438-7d67a8cf0169 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:42:47.098913 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-dc9dc9c6-1bff-42e4-8438-7d67a8cf0169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:47.142431 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15908c59-9e02-405f-9bd8-88454521d9ab admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:47.155265 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1300/2561] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:42:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:42:47.156088 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9dc9c6-1bff-42e4-8438-7d67a8cf0169 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:47.165536 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9dc9c6-1bff-42e4-8438-7d67a8cf0169 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:47.171063 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cf1045-0006-419b-b11f-3028a6b613e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:47.173380 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cf1045-0006-419b-b11f-3028a6b613e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:47.173380 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cf1045-0006-419b-b11f-3028a6b613e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:47.173380 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cf1045-0006-419b-b11f-3028a6b613e4 admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:47.173380 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cf1045-0006-419b-b11f-3028a6b613e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:47.174957 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cf1045-0006-419b-b11f-3028a6b613e4 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:47.200364 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2494266d-7ca1-4aac-9d51-fd85d2008f9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:47.201036 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2494266d-7ca1-4aac-9d51-fd85d2008f9b admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:47.201332 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2494266d-7ca1-4aac-9d51-fd85d2008f9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:47.201597 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2494266d-7ca1-4aac-9d51-fd85d2008f9b admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:47.202322 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2494266d-7ca1-4aac-9d51-fd85d2008f9b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:47.204251 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2494266d-7ca1-4aac-9d51-fd85d2008f9b admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:47.298122 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1301/2562] 10.222.0.66 () {60 vars in 1143 bytes} [Mon Jul 18 02:42:47 2022] POST /identity/v3/users => generated 287 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:42:47.298990 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cf1045-0006-419b-b11f-3028a6b613e4 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:47.303538 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1262/2563] 10.222.0.66 () {60 vars in 1143 bytes} [Mon Jul 18 02:42:47 2022] POST /identity/v3/users => generated 290 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:42:47.307148 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2494266d-7ca1-4aac-9d51-fd85d2008f9b admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:47.309640 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cf1045-0006-419b-b11f-3028a6b613e4 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:47.314809 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2f6d41-c9a7-4fea-98dc-c2125bab12a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:47.315477 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2f6d41-c9a7-4fea-98dc-c2125bab12a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:47.315765 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2f6d41-c9a7-4fea-98dc-c2125bab12a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:47.316041 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2f6d41-c9a7-4fea-98dc-c2125bab12a7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:47.316857 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2f6d41-c9a7-4fea-98dc-c2125bab12a7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:42:47.317081 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2f6d41-c9a7-4fea-98dc-c2125bab12a7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:47.318736 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2f6d41-c9a7-4fea-98dc-c2125bab12a7 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:47.319037 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2494266d-7ca1-4aac-9d51-fd85d2008f9b admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:47.325165 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca56e94-0840-49c4-bd4c-d4341d00c5cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:47.326320 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca56e94-0840-49c4-bd4c-d4341d00c5cb admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:47.326320 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca56e94-0840-49c4-bd4c-d4341d00c5cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:47.327931 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca56e94-0840-49c4-bd4c-d4341d00c5cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:47.327931 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca56e94-0840-49c4-bd4c-d4341d00c5cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:47.329259 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca56e94-0840-49c4-bd4c-d4341d00c5cb admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:47.331814 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1302/2564] 10.222.0.66 () {58 vars in 1152 bytes} [Mon Jul 18 02:42:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:42:47.332735 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2f6d41-c9a7-4fea-98dc-c2125bab12a7 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:47.345089 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1263/2565] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:42:47 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 02:42:47.345480 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2f6d41-c9a7-4fea-98dc-c2125bab12a7 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:47.346286 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca56e94-0840-49c4-bd4c-d4341d00c5cb admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:47.350881 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae74fbb8-af73-4fbd-b845-83fa43386630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:47.351554 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae74fbb8-af73-4fbd-b845-83fa43386630 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:47.351871 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae74fbb8-af73-4fbd-b845-83fa43386630 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:47.354239 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae74fbb8-af73-4fbd-b845-83fa43386630 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:47.354239 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae74fbb8-af73-4fbd-b845-83fa43386630 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:47.355497 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae74fbb8-af73-4fbd-b845-83fa43386630 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:47.358889 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca56e94-0840-49c4-bd4c-d4341d00c5cb admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:47.363680 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a4b76c-745a-4221-93a9-523a6caa1db3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:47.364389 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a4b76c-745a-4221-93a9-523a6caa1db3 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:47.364673 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a4b76c-745a-4221-93a9-523a6caa1db3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:47.364951 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a4b76c-745a-4221-93a9-523a6caa1db3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:47.365638 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a4b76c-745a-4221-93a9-523a6caa1db3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:47.365979 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1303/2566] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:42:47 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:42:47.366804 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae74fbb8-af73-4fbd-b845-83fa43386630 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:47.367677 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a4b76c-745a-4221-93a9-523a6caa1db3 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:47.377181 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae74fbb8-af73-4fbd-b845-83fa43386630 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:47.382265 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de541511-e427-4abb-b846-f09522a6a64a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:47.383268 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de541511-e427-4abb-b846-f09522a6a64a admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:47.383777 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de541511-e427-4abb-b846-f09522a6a64a admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:47.384170 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de541511-e427-4abb-b846-f09522a6a64a admin admin] PATH_INFO: `/v3/projects/92abdc2ac1db4022bf55caa96817617d/users/32e70797e1ae456e940eef098d843285/roles/d4aa3324f0c84887b41d0dc4acf7f30e` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:47.400186 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de541511-e427-4abb-b846-f09522a6a64a admin admin] RBAC: Authorizing `identity:create_grant(project_id=92abdc2ac1db4022bf55caa96817617d, user_id=32e70797e1ae456e940eef098d843285, role_id=d4aa3324f0c84887b41d0dc4acf7f30e)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:47.403638 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de541511-e427-4abb-b846-f09522a6a64a admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:47.468538 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1304/2567] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:42:47 2022] PUT /identity/v3/projects/92abdc2ac1db4022bf55caa96817617d/users/32e70797e1ae456e940eef098d843285/roles/d4aa3324f0c84887b41d0dc4acf7f30e => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:42:47.469500 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de541511-e427-4abb-b846-f09522a6a64a admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:47.469961 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1264/2568] 10.222.0.66 () {60 vars in 1152 bytes} [Mon Jul 18 02:42:47 2022] POST /identity/v3/projects => generated 373 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:42:47.472401 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a4b76c-745a-4221-93a9-523a6caa1db3 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:47.535061 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de541511-e427-4abb-b846-f09522a6a64a admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:47.535677 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a4b76c-745a-4221-93a9-523a6caa1db3 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:47.539788 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e249e61-4eb7-4071-a9f8-0de88d9ee70d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:47.540626 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e249e61-4eb7-4071-a9f8-0de88d9ee70d admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:47.540748 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7082ed8b-9d0a-4ed5-a14a-07172429af6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:47.540916 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e249e61-4eb7-4071-a9f8-0de88d9ee70d admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:47.541198 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e249e61-4eb7-4071-a9f8-0de88d9ee70d admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:47.541405 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7082ed8b-9d0a-4ed5-a14a-07172429af6a admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:47.541690 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7082ed8b-9d0a-4ed5-a14a-07172429af6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:47.541932 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e249e61-4eb7-4071-a9f8-0de88d9ee70d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:47.542045 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7082ed8b-9d0a-4ed5-a14a-07172429af6a admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:47.543340 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7082ed8b-9d0a-4ed5-a14a-07172429af6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:47.544495 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e249e61-4eb7-4071-a9f8-0de88d9ee70d admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:47.545142 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7082ed8b-9d0a-4ed5-a14a-07172429af6a admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:47.566541 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1305/2569] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:42:47 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:42:47.567148 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1265/2570] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:42:47 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 02:42:47.567269 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e249e61-4eb7-4071-a9f8-0de88d9ee70d admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:47.567380 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7082ed8b-9d0a-4ed5-a14a-07172429af6a admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:47.570431 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7082ed8b-9d0a-4ed5-a14a-07172429af6a admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:47.570556 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e249e61-4eb7-4071-a9f8-0de88d9ee70d admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:47.575565 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e525b4-0376-4cd2-b964-db33a26e8f1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:47.575756 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caa1cec-0eea-4d93-9484-4c63f2263c75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:47.576243 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e525b4-0376-4cd2-b964-db33a26e8f1c admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:47.576529 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e525b4-0376-4cd2-b964-db33a26e8f1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:47.576641 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caa1cec-0eea-4d93-9484-4c63f2263c75 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:47.576804 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e525b4-0376-4cd2-b964-db33a26e8f1c admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:47.576964 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caa1cec-0eea-4d93-9484-4c63f2263c75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:47.577243 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caa1cec-0eea-4d93-9484-4c63f2263c75 admin admin] PATH_INFO: `/v3/projects/2fe363fe43ef4fd9aec026c4cdc3e18e/users/681d0f989d7d459e83a32066fd5ed0a2/roles/65feabf3b27944a8b01a86c8ac608457` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:47.577497 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e525b4-0376-4cd2-b964-db33a26e8f1c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:47.579564 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e525b4-0376-4cd2-b964-db33a26e8f1c admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:47.593272 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8caa1cec-0eea-4d93-9484-4c63f2263c75 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2fe363fe43ef4fd9aec026c4cdc3e18e, user_id=681d0f989d7d459e83a32066fd5ed0a2, role_id=65feabf3b27944a8b01a86c8ac608457)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:47.596184 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8caa1cec-0eea-4d93-9484-4c63f2263c75 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:47.676560 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1306/2571] 10.222.0.66 () {60 vars in 1143 bytes} [Mon Jul 18 02:42:47 2022] POST /identity/v3/users => generated 287 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:42:47.676560 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e525b4-0376-4cd2-b964-db33a26e8f1c admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:47.683815 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e525b4-0376-4cd2-b964-db33a26e8f1c admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:47.688629 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481d55d2-c49e-45ca-8d66-d526d4852606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:47.689472 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481d55d2-c49e-45ca-8d66-d526d4852606 admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:47.689750 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481d55d2-c49e-45ca-8d66-d526d4852606 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:47.690018 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481d55d2-c49e-45ca-8d66-d526d4852606 admin admin] PATH_INFO: `/v3/projects/92abdc2ac1db4022bf55caa96817617d/users/32e70797e1ae456e940eef098d843285/roles/d4aa3324f0c84887b41d0dc4acf7f30e` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:47.692008 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481d55d2-c49e-45ca-8d66-d526d4852606 admin admin] RBAC: Authorizing `identity:create_grant(project_id=92abdc2ac1db4022bf55caa96817617d, user_id=32e70797e1ae456e940eef098d843285, role_id=d4aa3324f0c84887b41d0dc4acf7f30e)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:47.694690 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481d55d2-c49e-45ca-8d66-d526d4852606 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:47.704580 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1266/2572] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:42:47 2022] PUT /identity/v3/projects/2fe363fe43ef4fd9aec026c4cdc3e18e/users/681d0f989d7d459e83a32066fd5ed0a2/roles/65feabf3b27944a8b01a86c8ac608457 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 02:42:47.704580 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caa1cec-0eea-4d93-9484-4c63f2263c75 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:47.708370 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1307/2573] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:42:47 2022] PUT /identity/v3/projects/92abdc2ac1db4022bf55caa96817617d/users/32e70797e1ae456e940eef098d843285/roles/d4aa3324f0c84887b41d0dc4acf7f30e => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:42:47.709962 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71722e3-8852-41fc-9875-3d3e3cbca5d4 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:47.710454 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71722e3-8852-41fc-9875-3d3e3cbca5d4 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:47.710899 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71722e3-8852-41fc-9875-3d3e3cbca5d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:47.729038 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-b71722e3-8852-41fc-9875-3d3e3cbca5d4 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:42:47.739173 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-b71722e3-8852-41fc-9875-3d3e3cbca5d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:47.768168 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caa1cec-0eea-4d93-9484-4c63f2263c75 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:47.788346 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:42:47.796558 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1308/2574] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:42:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:42:47.796558 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da21b9e4-136e-48d0-bd2f-2c9a7712d43e None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:47.796558 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da21b9e4-136e-48d0-bd2f-2c9a7712d43e None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:47.796558 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da21b9e4-136e-48d0-bd2f-2c9a7712d43e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:47.834206 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-da21b9e4-136e-48d0-bd2f-2c9a7712d43e None None] MFA Rules not processed for user `32e70797e1ae456e940eef098d843285`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:42:47.834448 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8358536d-6b0b-4db6-8c78-0e37f4eccc05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:47.834448 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8358536d-6b0b-4db6-8c78-0e37f4eccc05 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:47.834689 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-da21b9e4-136e-48d0-bd2f-2c9a7712d43e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:47.834794 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8358536d-6b0b-4db6-8c78-0e37f4eccc05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:47.834794 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8358536d-6b0b-4db6-8c78-0e37f4eccc05 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:47.834794 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8358536d-6b0b-4db6-8c78-0e37f4eccc05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:47.834794 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8358536d-6b0b-4db6-8c78-0e37f4eccc05 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:47.844140 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1267/2575] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:42:47 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:42:47.844140 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8358536d-6b0b-4db6-8c78-0e37f4eccc05 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:47.844140 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-8358536d-6b0b-4db6-8c78-0e37f4eccc05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:47.872942 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:42:47.895107 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1309/2576] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:42:47 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:42:47.896479 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da21b9e4-136e-48d0-bd2f-2c9a7712d43e None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:47.904418 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8358536d-6b0b-4db6-8c78-0e37f4eccc05 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:47.907334 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da21b9e4-136e-48d0-bd2f-2c9a7712d43e None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:47.908390 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede4c22c-df2b-44a8-8cd1-f6eef4653de8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eaf19a259c2741cea375a38763e33cdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:47.909042 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede4c22c-df2b-44a8-8cd1-f6eef4653de8 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:47.909334 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede4c22c-df2b-44a8-8cd1-f6eef4653de8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:47.909616 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede4c22c-df2b-44a8-8cd1-f6eef4653de8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:47.912376 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc42c605-032e-4aee-ad0d-99578de5bfcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:47.913037 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc42c605-032e-4aee-ad0d-99578de5bfcf admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:47.913322 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc42c605-032e-4aee-ad0d-99578de5bfcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:47.913598 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc42c605-032e-4aee-ad0d-99578de5bfcf admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:47.914418 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc42c605-032e-4aee-ad0d-99578de5bfcf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:47.916015 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc42c605-032e-4aee-ad0d-99578de5bfcf admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:47.919693 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede4c22c-df2b-44a8-8cd1-f6eef4653de8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:47.921162 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede4c22c-df2b-44a8-8cd1-f6eef4653de8 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:47.926080 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1310/2577] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:42:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:42:47.927207 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc42c605-032e-4aee-ad0d-99578de5bfcf admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:47.929181 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-dc42c605-032e-4aee-ad0d-99578de5bfcf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:47.932024 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1268/2578] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:42:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:42:47.935071 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede4c22c-df2b-44a8-8cd1-f6eef4653de8 service neutron] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:47.943452 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede4c22c-df2b-44a8-8cd1-f6eef4653de8 service neutron] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:47.949198 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98964271-7264-450c-aaeb-7f76807b3b31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:47.950243 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98964271-7264-450c-aaeb-7f76807b3b31 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:47.950697 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98964271-7264-450c-aaeb-7f76807b3b31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:47.951284 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98964271-7264-450c-aaeb-7f76807b3b31 admin admin] PATH_INFO: `/v3/projects/c048f3a4c111472db2a3d1cded862ed0/users/81f2f385e5b248d38267eba820765c0f/roles/65feabf3b27944a8b01a86c8ac608457` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:47.968004 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98964271-7264-450c-aaeb-7f76807b3b31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c048f3a4c111472db2a3d1cded862ed0, user_id=81f2f385e5b248d38267eba820765c0f, role_id=65feabf3b27944a8b01a86c8ac608457)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:47.970795 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98964271-7264-450c-aaeb-7f76807b3b31 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:47.989018 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc42c605-032e-4aee-ad0d-99578de5bfcf admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:47.992732 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3da9cf9-4f8e-4251-a20d-3142e328c3a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd49388facdf446885e55b45ef91f01e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:47.993377 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3da9cf9-4f8e-4251-a20d-3142e328c3a7 service cinder] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:47.993664 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3da9cf9-4f8e-4251-a20d-3142e328c3a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:47.993945 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3da9cf9-4f8e-4251-a20d-3142e328c3a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:48.003747 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3da9cf9-4f8e-4251-a20d-3142e328c3a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:48.005173 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3da9cf9-4f8e-4251-a20d-3142e328c3a7 service cinder] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:48.015247 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1311/2579] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:42:47 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:42:48.069512 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1269/2580] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:42:47 2022] PUT /identity/v3/projects/c048f3a4c111472db2a3d1cded862ed0/users/81f2f385e5b248d38267eba820765c0f/roles/65feabf3b27944a8b01a86c8ac608457 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:42:48.080584 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9415a4e1-bd1c-4cd4-9b84-94595f41a2dd None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:48.080880 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9415a4e1-bd1c-4cd4-9b84-94595f41a2dd None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:48.081160 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9415a4e1-bd1c-4cd4-9b84-94595f41a2dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:48.108584 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-9415a4e1-bd1c-4cd4-9b84-94595f41a2dd None None] MFA Rules not processed for user `81f2f385e5b248d38267eba820765c0f`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:42:48.109721 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-9415a4e1-bd1c-4cd4-9b84-94595f41a2dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:48.174581 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1312/2581] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:42:48 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:42:48.179024 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98964271-7264-450c-aaeb-7f76807b3b31 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:48.229633 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:42:48.243748 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98964271-7264-450c-aaeb-7f76807b3b31 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:48.249339 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64aaccf6-ae4c-423e-af00-7210908b9eae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:48.250006 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aaccf6-ae4c-423e-af00-7210908b9eae admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:48.250336 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aaccf6-ae4c-423e-af00-7210908b9eae admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:48.250614 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aaccf6-ae4c-423e-af00-7210908b9eae admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:48.251311 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64aaccf6-ae4c-423e-af00-7210908b9eae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:48.253501 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64aaccf6-ae4c-423e-af00-7210908b9eae admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:48.315054 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:42:48.365169 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1270/2582] 10.222.0.66 () {60 vars in 1152 bytes} [Mon Jul 18 02:42:48 2022] POST /identity/v3/projects => generated 373 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:42:48.368156 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9415a4e1-bd1c-4cd4-9b84-94595f41a2dd None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:48.379698 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9415a4e1-bd1c-4cd4-9b84-94595f41a2dd None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:48.439318 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65aca93b-a194-440f-8692-452f7511304d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:48.439991 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aca93b-a194-440f-8692-452f7511304d admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:48.440275 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aca93b-a194-440f-8692-452f7511304d admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:48.440548 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aca93b-a194-440f-8692-452f7511304d admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:48.441228 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65aca93b-a194-440f-8692-452f7511304d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:48.443166 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65aca93b-a194-440f-8692-452f7511304d admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:48.557067 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1313/2583] 10.222.0.66 () {60 vars in 1143 bytes} [Mon Jul 18 02:42:48 2022] POST /identity/v3/users => generated 293 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:42:48.565324 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64aaccf6-ae4c-423e-af00-7210908b9eae admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:48.576915 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64aaccf6-ae4c-423e-af00-7210908b9eae admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:48.581685 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165cf182-51d2-4335-89f9-72df2171850a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:48.582887 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165cf182-51d2-4335-89f9-72df2171850a admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:48.582887 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165cf182-51d2-4335-89f9-72df2171850a admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:48.583092 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165cf182-51d2-4335-89f9-72df2171850a admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:48.583649 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165cf182-51d2-4335-89f9-72df2171850a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:48.585268 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165cf182-51d2-4335-89f9-72df2171850a admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:48.597145 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1271/2584] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:42:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:42:48.601478 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65aca93b-a194-440f-8692-452f7511304d admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:48.611761 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65aca93b-a194-440f-8692-452f7511304d admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:48.616755 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55405590-505c-4128-90d6-1fe38a229a18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:48.617582 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55405590-505c-4128-90d6-1fe38a229a18 admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:48.617863 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55405590-505c-4128-90d6-1fe38a229a18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:48.618174 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55405590-505c-4128-90d6-1fe38a229a18 admin admin] PATH_INFO: `/v3/projects/7d33beaa0fa641e58c501e7311caf194/users/00b8bd7a6a814fada57f9062b1571f35/roles/d4aa3324f0c84887b41d0dc4acf7f30e` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:48.623562 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:42:48.633569 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55405590-505c-4128-90d6-1fe38a229a18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d33beaa0fa641e58c501e7311caf194, user_id=00b8bd7a6a814fada57f9062b1571f35, role_id=d4aa3324f0c84887b41d0dc4acf7f30e)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:48.636879 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55405590-505c-4128-90d6-1fe38a229a18 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:48.772707 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1314/2585] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:42:48 2022] PUT /identity/v3/projects/7d33beaa0fa641e58c501e7311caf194/users/00b8bd7a6a814fada57f9062b1571f35/roles/d4aa3324f0c84887b41d0dc4acf7f30e => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:42:48.777423 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165cf182-51d2-4335-89f9-72df2171850a admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:48.787691 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165cf182-51d2-4335-89f9-72df2171850a admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:48.835427 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:42:48.846820 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12854def-9aae-4cd8-95ab-55be4e0f6ea0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:48.847560 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12854def-9aae-4cd8-95ab-55be4e0f6ea0 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:48.847850 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12854def-9aae-4cd8-95ab-55be4e0f6ea0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:48.848128 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12854def-9aae-4cd8-95ab-55be4e0f6ea0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:48.848862 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12854def-9aae-4cd8-95ab-55be4e0f6ea0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:48.850634 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12854def-9aae-4cd8-95ab-55be4e0f6ea0 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:48.863000 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1272/2586] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:42:48 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:42:48.869818 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55405590-505c-4128-90d6-1fe38a229a18 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:48.882688 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55405590-505c-4128-90d6-1fe38a229a18 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:48.887839 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e0a174-e9c0-414c-87e0-1cee13b3e759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:48.888714 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e0a174-e9c0-414c-87e0-1cee13b3e759 admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:48.888993 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e0a174-e9c0-414c-87e0-1cee13b3e759 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:48.889265 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e0a174-e9c0-414c-87e0-1cee13b3e759 admin admin] PATH_INFO: `/v3/projects/7d33beaa0fa641e58c501e7311caf194/users/00b8bd7a6a814fada57f9062b1571f35/roles/d4aa3324f0c84887b41d0dc4acf7f30e` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:48.891289 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e0a174-e9c0-414c-87e0-1cee13b3e759 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d33beaa0fa641e58c501e7311caf194, user_id=00b8bd7a6a814fada57f9062b1571f35, role_id=d4aa3324f0c84887b41d0dc4acf7f30e)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:48.893905 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e0a174-e9c0-414c-87e0-1cee13b3e759 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:48.904437 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1315/2587] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:42:48 2022] PUT /identity/v3/projects/7d33beaa0fa641e58c501e7311caf194/users/00b8bd7a6a814fada57f9062b1571f35/roles/d4aa3324f0c84887b41d0dc4acf7f30e => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:42:48.915768 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616e47af-1587-4e8c-af9b-4ed42270c93b None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:48.916064 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616e47af-1587-4e8c-af9b-4ed42270c93b None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:48.916353 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616e47af-1587-4e8c-af9b-4ed42270c93b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:48.943921 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-616e47af-1587-4e8c-af9b-4ed42270c93b None None] MFA Rules not processed for user `00b8bd7a6a814fada57f9062b1571f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:42:48.945134 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-616e47af-1587-4e8c-af9b-4ed42270c93b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:49.080357 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1273/2588] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:42:48 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 02:42:49.092287 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e0a174-e9c0-414c-87e0-1cee13b3e759 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:49.094150 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-c6e0a174-e9c0-414c-87e0-1cee13b3e759 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:49.151939 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e0a174-e9c0-414c-87e0-1cee13b3e759 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:49.156589 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e2c6ac-8ada-4a18-bea1-f778936d20e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd49388facdf446885e55b45ef91f01e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:49.156712 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e2c6ac-8ada-4a18-bea1-f778936d20e8 service cinder] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:49.156930 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e2c6ac-8ada-4a18-bea1-f778936d20e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:49.157214 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e2c6ac-8ada-4a18-bea1-f778936d20e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:49.206544 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:42:49.220327 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e2c6ac-8ada-4a18-bea1-f778936d20e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:49.221843 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e2c6ac-8ada-4a18-bea1-f778936d20e8 service cinder] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:49.231168 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1316/2589] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:42:49 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 02:42:49.347070 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:42:50.226553 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:42:50.445241 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:42:50.720549 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:42:51.865562 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:42:51.989727 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616e47af-1587-4e8c-af9b-4ed42270c93b None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:52.064288 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616e47af-1587-4e8c-af9b-4ed42270c93b None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:52.077440 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9adeebb-4458-464b-ba83-98095181c826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:52.078680 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9adeebb-4458-464b-ba83-98095181c826 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:52.079154 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9adeebb-4458-464b-ba83-98095181c826 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:52.079678 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9adeebb-4458-464b-ba83-98095181c826 admin admin] PATH_INFO: `/v3/users/1b332bd700fc48efa70715c13a0ba6a9` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:52.080961 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9adeebb-4458-464b-ba83-98095181c826 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b332bd700fc48efa70715c13a0ba6a9)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:52.082967 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9adeebb-4458-464b-ba83-98095181c826 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:52.424239 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-b9adeebb-4458-464b-ba83-98095181c826 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b332bd700fc48efa70715c13a0ba6a9'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:52.492253 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-b9adeebb-4458-464b-ba83-98095181c826 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b332bd700fc48efa70715c13a0ba6a9'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:52.493221 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-b9adeebb-4458-464b-ba83-98095181c826 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b332bd700fc48efa70715c13a0ba6a9'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:52.511592 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-b9adeebb-4458-464b-ba83-98095181c826 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b332bd700fc48efa70715c13a0ba6a9'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:52.512518 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-b9adeebb-4458-464b-ba83-98095181c826 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b332bd700fc48efa70715c13a0ba6a9'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:52.527561 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1274/2590] 10.222.0.66 () {58 vars in 1223 bytes} [Mon Jul 18 02:42:51 2022] DELETE /identity/v3/users/1b332bd700fc48efa70715c13a0ba6a9 => generated 0 bytes in 538 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:42:52.532753 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c825d4b5-5edf-4424-a14b-318d142f7195 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:52.533244 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c825d4b5-5edf-4424-a14b-318d142f7195 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:52.533244 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c825d4b5-5edf-4424-a14b-318d142f7195 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:52.559505 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-c825d4b5-5edf-4424-a14b-318d142f7195 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:42:52.561280 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-c825d4b5-5edf-4424-a14b-318d142f7195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:52.629405 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1317/2591] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:42:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:42:52.640356 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9adeebb-4458-464b-ba83-98095181c826 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:52.641825 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-b9adeebb-4458-464b-ba83-98095181c826 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:52.679445 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c825d4b5-5edf-4424-a14b-318d142f7195 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:52.683445 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-c825d4b5-5edf-4424-a14b-318d142f7195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:52.708681 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9adeebb-4458-464b-ba83-98095181c826 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:52.712356 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c825d4b5-5edf-4424-a14b-318d142f7195 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:52.712617 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8a1148-08d8-45a8-9e67-902b6d6259fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eaf19a259c2741cea375a38763e33cdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:52.713696 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8a1148-08d8-45a8-9e67-902b6d6259fa service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:52.714080 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8a1148-08d8-45a8-9e67-902b6d6259fa service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:52.714588 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8a1148-08d8-45a8-9e67-902b6d6259fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:52.719622 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a6c4a0-e543-4c5f-a83e-00aac61ed221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:52.722448 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a6c4a0-e543-4c5f-a83e-00aac61ed221 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:52.722448 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a6c4a0-e543-4c5f-a83e-00aac61ed221 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:52.722448 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a6c4a0-e543-4c5f-a83e-00aac61ed221 admin admin] PATH_INFO: `/v3/users/81f2f385e5b248d38267eba820765c0f` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:52.722448 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a6c4a0-e543-4c5f-a83e-00aac61ed221 admin admin] RBAC: Authorizing `identity:delete_user(user_id=81f2f385e5b248d38267eba820765c0f)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:52.724288 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a6c4a0-e543-4c5f-a83e-00aac61ed221 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:52.729782 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8a1148-08d8-45a8-9e67-902b6d6259fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:52.732272 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8a1148-08d8-45a8-9e67-902b6d6259fa service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:52.748104 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1275/2592] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:42:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 02:42:53.056902 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-a5a6c4a0-e543-4c5f-a83e-00aac61ed221 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81f2f385e5b248d38267eba820765c0f'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:53.130349 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-a5a6c4a0-e543-4c5f-a83e-00aac61ed221 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81f2f385e5b248d38267eba820765c0f'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:53.146747 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-a5a6c4a0-e543-4c5f-a83e-00aac61ed221 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81f2f385e5b248d38267eba820765c0f'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:53.149561 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-a5a6c4a0-e543-4c5f-a83e-00aac61ed221 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81f2f385e5b248d38267eba820765c0f'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:53.174576 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-a5a6c4a0-e543-4c5f-a83e-00aac61ed221 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81f2f385e5b248d38267eba820765c0f'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:53.184118 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1318/2593] 10.222.0.66 () {58 vars in 1223 bytes} [Mon Jul 18 02:42:52 2022] DELETE /identity/v3/users/81f2f385e5b248d38267eba820765c0f => generated 0 bytes in 502 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:42:53.186177 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f5a01e-629c-429a-a2eb-95623d9b9ee5 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:53.186670 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f5a01e-629c-429a-a2eb-95623d9b9ee5 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:53.186799 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f5a01e-629c-429a-a2eb-95623d9b9ee5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:53.212801 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-44f5a01e-629c-429a-a2eb-95623d9b9ee5 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:42:53.214904 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-44f5a01e-629c-429a-a2eb-95623d9b9ee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:53.291124 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1276/2594] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:42:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:42:53.303340 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a6c4a0-e543-4c5f-a83e-00aac61ed221 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:53.304801 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-a5a6c4a0-e543-4c5f-a83e-00aac61ed221 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:53.397089 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a6c4a0-e543-4c5f-a83e-00aac61ed221 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:53.402875 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddd7b3f-3ef2-4593-b935-501798b54d04 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eaf19a259c2741cea375a38763e33cdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:53.402875 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddd7b3f-3ef2-4593-b935-501798b54d04 service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:53.402875 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddd7b3f-3ef2-4593-b935-501798b54d04 service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:53.402875 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddd7b3f-3ef2-4593-b935-501798b54d04 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:53.413684 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddd7b3f-3ef2-4593-b935-501798b54d04 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:53.415289 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddd7b3f-3ef2-4593-b935-501798b54d04 service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:53.427565 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1319/2595] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:42:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:42:53.508828 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f5a01e-629c-429a-a2eb-95623d9b9ee5 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:53.510406 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-44f5a01e-629c-429a-a2eb-95623d9b9ee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:53.554788 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f5a01e-629c-429a-a2eb-95623d9b9ee5 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:53.562669 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fb6691-ca74-4b25-b5c6-707f71f3679c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:53.563702 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fb6691-ca74-4b25-b5c6-707f71f3679c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:53.563988 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fb6691-ca74-4b25-b5c6-707f71f3679c admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:53.564650 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fb6691-ca74-4b25-b5c6-707f71f3679c admin admin] PATH_INFO: `/v3/projects/670634a51ed94c67beaf0d1b4ffc38e9` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:53.566523 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fb6691-ca74-4b25-b5c6-707f71f3679c admin admin] RBAC: Authorizing `identity:delete_project(project_id=670634a51ed94c67beaf0d1b4ffc38e9)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:53.569089 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fb6691-ca74-4b25-b5c6-707f71f3679c admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:53.697572 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-e7fb6691-ca74-4b25-b5c6-707f71f3679c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '670634a51ed94c67beaf0d1b4ffc38e9', 'request_id': 'req-e7fb6691-ca74-4b25-b5c6-707f71f3679c'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:53.705689 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-e7fb6691-ca74-4b25-b5c6-707f71f3679c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '670634a51ed94c67beaf0d1b4ffc38e9', 'request_id': 'req-e7fb6691-ca74-4b25-b5c6-707f71f3679c'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:53.713887 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-e7fb6691-ca74-4b25-b5c6-707f71f3679c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '670634a51ed94c67beaf0d1b4ffc38e9', 'request_id': 'req-e7fb6691-ca74-4b25-b5c6-707f71f3679c'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:53.856612 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:42:53.872115 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-e7fb6691-ca74-4b25-b5c6-707f71f3679c admin admin] The token cache is being invalidate because project 670634a51ed94c67beaf0d1b4ffc38e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 02:42:53.872469 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-e7fb6691-ca74-4b25-b5c6-707f71f3679c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:53.873915 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-e7fb6691-ca74-4b25-b5c6-707f71f3679c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:53.879092 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1277/2596] 10.222.0.66 () {58 vars in 1232 bytes} [Mon Jul 18 02:42:53 2022] DELETE /identity/v3/projects/670634a51ed94c67beaf0d1b4ffc38e9 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:42:53.880467 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddd7b3f-3ef2-4593-b935-501798b54d04 service neutron] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:53.890340 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:42:53.891995 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddd7b3f-3ef2-4593-b935-501798b54d04 service neutron] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:53.981581 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e686e1-7423-4eb6-8a56-be23f65b98ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:53.981581 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e686e1-7423-4eb6-8a56-be23f65b98ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:53.981581 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e686e1-7423-4eb6-8a56-be23f65b98ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:53.981581 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e686e1-7423-4eb6-8a56-be23f65b98ca admin admin] PATH_INFO: `/v3/users/32e70797e1ae456e940eef098d843285` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:53.982946 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e686e1-7423-4eb6-8a56-be23f65b98ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=32e70797e1ae456e940eef098d843285)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:53.985393 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e686e1-7423-4eb6-8a56-be23f65b98ca admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:54.053532 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fb6691-ca74-4b25-b5c6-707f71f3679c admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:54.059325 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-e7fb6691-ca74-4b25-b5c6-707f71f3679c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:54.075549 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fb6691-ca74-4b25-b5c6-707f71f3679c admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:54.081434 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e66143-c406-4d2b-9aa2-b2c15b7949fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:54.082443 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e66143-c406-4d2b-9aa2-b2c15b7949fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:54.082796 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e66143-c406-4d2b-9aa2-b2c15b7949fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:54.083244 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e66143-c406-4d2b-9aa2-b2c15b7949fd admin admin] PATH_INFO: `/v3/projects/c048f3a4c111472db2a3d1cded862ed0` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:54.085416 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e66143-c406-4d2b-9aa2-b2c15b7949fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=c048f3a4c111472db2a3d1cded862ed0)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:54.088141 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e66143-c406-4d2b-9aa2-b2c15b7949fd admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:54.198745 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-49e686e1-7423-4eb6-8a56-be23f65b98ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32e70797e1ae456e940eef098d843285'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:54.250910 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-72e66143-c406-4d2b-9aa2-b2c15b7949fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c048f3a4c111472db2a3d1cded862ed0', 'request_id': 'req-72e66143-c406-4d2b-9aa2-b2c15b7949fd'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:54.258649 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-72e66143-c406-4d2b-9aa2-b2c15b7949fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c048f3a4c111472db2a3d1cded862ed0', 'request_id': 'req-72e66143-c406-4d2b-9aa2-b2c15b7949fd'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:54.268989 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-72e66143-c406-4d2b-9aa2-b2c15b7949fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c048f3a4c111472db2a3d1cded862ed0', 'request_id': 'req-72e66143-c406-4d2b-9aa2-b2c15b7949fd'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:54.272844 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-49e686e1-7423-4eb6-8a56-be23f65b98ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32e70797e1ae456e940eef098d843285'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:54.281946 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-49e686e1-7423-4eb6-8a56-be23f65b98ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32e70797e1ae456e940eef098d843285'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:54.283475 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-49e686e1-7423-4eb6-8a56-be23f65b98ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32e70797e1ae456e940eef098d843285'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:54.298918 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-49e686e1-7423-4eb6-8a56-be23f65b98ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32e70797e1ae456e940eef098d843285'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:54.307521 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1320/2597] 10.222.0.66 () {58 vars in 1223 bytes} [Mon Jul 18 02:42:53 2022] DELETE /identity/v3/users/32e70797e1ae456e940eef098d843285 => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:42:54.309002 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e686e1-7423-4eb6-8a56-be23f65b98ca admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:54.319571 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-49e686e1-7423-4eb6-8a56-be23f65b98ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:54.346023 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-72e66143-c406-4d2b-9aa2-b2c15b7949fd admin admin] The token cache is being invalidate because project c048f3a4c111472db2a3d1cded862ed0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 02:42:54.351342 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-72e66143-c406-4d2b-9aa2-b2c15b7949fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:54.351342 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-72e66143-c406-4d2b-9aa2-b2c15b7949fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:54.351342 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1278/2598] 10.222.0.66 () {58 vars in 1232 bytes} [Mon Jul 18 02:42:54 2022] DELETE /identity/v3/projects/c048f3a4c111472db2a3d1cded862ed0 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:42:54.352892 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e66143-c406-4d2b-9aa2-b2c15b7949fd admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:54.354547 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-72e66143-c406-4d2b-9aa2-b2c15b7949fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:54.454347 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e686e1-7423-4eb6-8a56-be23f65b98ca admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:54.460041 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d621b1f6-29da-4a9d-950e-c2ca6bae1a4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:54.461140 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d621b1f6-29da-4a9d-950e-c2ca6bae1a4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:54.461732 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d621b1f6-29da-4a9d-950e-c2ca6bae1a4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:54.462101 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d621b1f6-29da-4a9d-950e-c2ca6bae1a4a admin admin] PATH_INFO: `/v3/users/608f32f746dc4bdd873d9c182a5d6488` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:54.463536 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d621b1f6-29da-4a9d-950e-c2ca6bae1a4a admin admin] RBAC: Authorizing `identity:delete_user(user_id=608f32f746dc4bdd873d9c182a5d6488)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:54.465622 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d621b1f6-29da-4a9d-950e-c2ca6bae1a4a admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:54.479625 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e66143-c406-4d2b-9aa2-b2c15b7949fd admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:54.487747 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc6a07a-a074-4f5a-91e5-36f01df1f4ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:54.488839 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc6a07a-a074-4f5a-91e5-36f01df1f4ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:54.489363 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc6a07a-a074-4f5a-91e5-36f01df1f4ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:54.489745 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc6a07a-a074-4f5a-91e5-36f01df1f4ca admin admin] PATH_INFO: `/v3/users/00b8bd7a6a814fada57f9062b1571f35` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:54.491376 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc6a07a-a074-4f5a-91e5-36f01df1f4ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=00b8bd7a6a814fada57f9062b1571f35)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:54.493529 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc6a07a-a074-4f5a-91e5-36f01df1f4ca admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:54.678220 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-d621b1f6-29da-4a9d-950e-c2ca6bae1a4a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '608f32f746dc4bdd873d9c182a5d6488'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:54.716512 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-abc6a07a-a074-4f5a-91e5-36f01df1f4ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00b8bd7a6a814fada57f9062b1571f35'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:54.760005 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-d621b1f6-29da-4a9d-950e-c2ca6bae1a4a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '608f32f746dc4bdd873d9c182a5d6488'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:54.767713 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-d621b1f6-29da-4a9d-950e-c2ca6bae1a4a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '608f32f746dc4bdd873d9c182a5d6488'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:54.768351 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-d621b1f6-29da-4a9d-950e-c2ca6bae1a4a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '608f32f746dc4bdd873d9c182a5d6488'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:54.780560 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-d621b1f6-29da-4a9d-950e-c2ca6bae1a4a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '608f32f746dc4bdd873d9c182a5d6488'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:54.786624 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1321/2599] 10.222.0.66 () {58 vars in 1223 bytes} [Mon Jul 18 02:42:54 2022] DELETE /identity/v3/users/608f32f746dc4bdd873d9c182a5d6488 => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:42:54.787982 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9a5d30-a2fa-4181-9a48-941f218a9189 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:54.788271 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9a5d30-a2fa-4181-9a48-941f218a9189 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:54.788554 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9a5d30-a2fa-4181-9a48-941f218a9189 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:54.816513 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-2c9a5d30-a2fa-4181-9a48-941f218a9189 None None] MFA Rules not processed for user `681d0f989d7d459e83a32066fd5ed0a2`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:42:54.817773 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-2c9a5d30-a2fa-4181-9a48-941f218a9189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:54.850311 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-abc6a07a-a074-4f5a-91e5-36f01df1f4ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00b8bd7a6a814fada57f9062b1571f35'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:54.850997 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-abc6a07a-a074-4f5a-91e5-36f01df1f4ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00b8bd7a6a814fada57f9062b1571f35'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:54.865075 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-abc6a07a-a074-4f5a-91e5-36f01df1f4ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00b8bd7a6a814fada57f9062b1571f35'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:54.865503 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-abc6a07a-a074-4f5a-91e5-36f01df1f4ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00b8bd7a6a814fada57f9062b1571f35'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:54.879162 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1279/2600] 10.222.0.66 () {58 vars in 1223 bytes} [Mon Jul 18 02:42:54 2022] DELETE /identity/v3/users/00b8bd7a6a814fada57f9062b1571f35 => generated 0 bytes in 527 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:42:54.880605 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f21f7c-75f0-4588-9e89-08a95c06b6ec None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:54.880918 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f21f7c-75f0-4588-9e89-08a95c06b6ec None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:54.881176 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f21f7c-75f0-4588-9e89-08a95c06b6ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:54.894077 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1322/2601] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:42:54 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:42:54.894964 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9a5d30-a2fa-4181-9a48-941f218a9189 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:54.898550 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-2c9a5d30-a2fa-4181-9a48-941f218a9189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:54.903322 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-09f21f7c-75f0-4588-9e89-08a95c06b6ec None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:42:54.905155 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-09f21f7c-75f0-4588-9e89-08a95c06b6ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:54.971420 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9a5d30-a2fa-4181-9a48-941f218a9189 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:54.972289 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1280/2602] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:42:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:42:54.973273 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f21f7c-75f0-4588-9e89-08a95c06b6ec None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:54.974775 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-09f21f7c-75f0-4588-9e89-08a95c06b6ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:54.982373 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec7c44e-7e0b-4c98-bcf5-30fbcd5bb414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:54.983336 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec7c44e-7e0b-4c98-bcf5-30fbcd5bb414 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:54.983624 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec7c44e-7e0b-4c98-bcf5-30fbcd5bb414 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:54.984314 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec7c44e-7e0b-4c98-bcf5-30fbcd5bb414 admin admin] PATH_INFO: `/v3/projects/92abdc2ac1db4022bf55caa96817617d` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:54.986230 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec7c44e-7e0b-4c98-bcf5-30fbcd5bb414 admin admin] RBAC: Authorizing `identity:delete_project(project_id=92abdc2ac1db4022bf55caa96817617d)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:54.988489 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f21f7c-75f0-4588-9e89-08a95c06b6ec None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:54.988622 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec7c44e-7e0b-4c98-bcf5-30fbcd5bb414 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:54.993615 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22180984-2a9b-4543-93d5-1337a6fbe57e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:54.994541 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22180984-2a9b-4543-93d5-1337a6fbe57e admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:54.994890 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22180984-2a9b-4543-93d5-1337a6fbe57e admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:54.995347 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22180984-2a9b-4543-93d5-1337a6fbe57e admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:54.996133 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22180984-2a9b-4543-93d5-1337a6fbe57e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:54.998157 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22180984-2a9b-4543-93d5-1337a6fbe57e admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:55.143956 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1281/2603] 10.222.0.66 () {60 vars in 1152 bytes} [Mon Jul 18 02:42:54 2022] POST /identity/v3/projects => generated 371 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:42:55.143956 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22180984-2a9b-4543-93d5-1337a6fbe57e admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:55.145832 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-22180984-2a9b-4543-93d5-1337a6fbe57e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:55.156360 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-aec7c44e-7e0b-4c98-bcf5-30fbcd5bb414 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92abdc2ac1db4022bf55caa96817617d', 'request_id': 'req-aec7c44e-7e0b-4c98-bcf5-30fbcd5bb414'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:55.168313 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-aec7c44e-7e0b-4c98-bcf5-30fbcd5bb414 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92abdc2ac1db4022bf55caa96817617d', 'request_id': 'req-aec7c44e-7e0b-4c98-bcf5-30fbcd5bb414'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:55.214828 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22180984-2a9b-4543-93d5-1337a6fbe57e admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:55.219241 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef96f07-1805-48d8-aa9f-7ccda08f74f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eaf19a259c2741cea375a38763e33cdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:55.219900 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef96f07-1805-48d8-aa9f-7ccda08f74f1 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:55.220184 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef96f07-1805-48d8-aa9f-7ccda08f74f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:55.220463 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef96f07-1805-48d8-aa9f-7ccda08f74f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:55.255649 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-aec7c44e-7e0b-4c98-bcf5-30fbcd5bb414 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92abdc2ac1db4022bf55caa96817617d', 'request_id': 'req-aec7c44e-7e0b-4c98-bcf5-30fbcd5bb414'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:55.269491 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-aec7c44e-7e0b-4c98-bcf5-30fbcd5bb414 admin admin] The token cache is being invalidate because project 92abdc2ac1db4022bf55caa96817617d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 02:42:55.269787 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-aec7c44e-7e0b-4c98-bcf5-30fbcd5bb414 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:55.270492 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-aec7c44e-7e0b-4c98-bcf5-30fbcd5bb414 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:55.272579 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1323/2604] 10.222.0.66 () {58 vars in 1232 bytes} [Mon Jul 18 02:42:54 2022] DELETE /identity/v3/projects/92abdc2ac1db4022bf55caa96817617d => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:42:55.273498 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec7c44e-7e0b-4c98-bcf5-30fbcd5bb414 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:55.274796 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-aec7c44e-7e0b-4c98-bcf5-30fbcd5bb414 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:55.290298 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef96f07-1805-48d8-aa9f-7ccda08f74f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:55.291740 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef96f07-1805-48d8-aa9f-7ccda08f74f1 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:55.301717 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1282/2605] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:42:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:42:55.303055 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a56c0b-1cc0-429f-92c9-2cdc66bf96e5 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:55.303347 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a56c0b-1cc0-429f-92c9-2cdc66bf96e5 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:55.303623 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a56c0b-1cc0-429f-92c9-2cdc66bf96e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:55.323667 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-c3a56c0b-1cc0-429f-92c9-2cdc66bf96e5 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:42:55.324820 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-c3a56c0b-1cc0-429f-92c9-2cdc66bf96e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:55.337444 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1283/2606] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:42:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:42:55.345750 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f77b9b3-4fec-4ce8-a5a5-08e1b82dac52 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:55.346036 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f77b9b3-4fec-4ce8-a5a5-08e1b82dac52 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:55.346530 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f77b9b3-4fec-4ce8-a5a5-08e1b82dac52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:55.368849 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-0f77b9b3-4fec-4ce8-a5a5-08e1b82dac52 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:42:55.369012 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec7c44e-7e0b-4c98-bcf5-30fbcd5bb414 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:55.371123 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-0f77b9b3-4fec-4ce8-a5a5-08e1b82dac52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:55.378808 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc9f8f1-7912-4e57-a6cb-e909bea1b4cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:55.379733 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc9f8f1-7912-4e57-a6cb-e909bea1b4cf admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:55.380116 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc9f8f1-7912-4e57-a6cb-e909bea1b4cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:55.380467 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc9f8f1-7912-4e57-a6cb-e909bea1b4cf admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:55.381477 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc9f8f1-7912-4e57-a6cb-e909bea1b4cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:55.381592 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1284/2607] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:42:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:42:55.383538 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc9f8f1-7912-4e57-a6cb-e909bea1b4cf admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:55.385956 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f77b9b3-4fec-4ce8-a5a5-08e1b82dac52 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:55.387438 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-0f77b9b3-4fec-4ce8-a5a5-08e1b82dac52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:55.400935 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f77b9b3-4fec-4ce8-a5a5-08e1b82dac52 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:55.407949 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44b8b10-24b1-41fc-96b5-0b8d7e66f4ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:55.409544 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44b8b10-24b1-41fc-96b5-0b8d7e66f4ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:55.409839 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44b8b10-24b1-41fc-96b5-0b8d7e66f4ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:55.410249 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44b8b10-24b1-41fc-96b5-0b8d7e66f4ff admin admin] PATH_INFO: `/v3/projects/7d33beaa0fa641e58c501e7311caf194` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:55.411409 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44b8b10-24b1-41fc-96b5-0b8d7e66f4ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d33beaa0fa641e58c501e7311caf194)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:55.415501 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44b8b10-24b1-41fc-96b5-0b8d7e66f4ff admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:55.486769 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1324/2608] 10.222.0.66 () {60 vars in 1143 bytes} [Mon Jul 18 02:42:55 2022] POST /identity/v3/users => generated 292 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:42:55.487633 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc9f8f1-7912-4e57-a6cb-e909bea1b4cf admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:55.497457 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc9f8f1-7912-4e57-a6cb-e909bea1b4cf admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:55.503335 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83488df1-55f4-42f9-932d-5c346785bdcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:55.504119 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83488df1-55f4-42f9-932d-5c346785bdcf admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:55.504408 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83488df1-55f4-42f9-932d-5c346785bdcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:55.504689 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83488df1-55f4-42f9-932d-5c346785bdcf admin admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:55.505426 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83488df1-55f4-42f9-932d-5c346785bdcf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:42:55.506113 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83488df1-55f4-42f9-932d-5c346785bdcf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:55.507420 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83488df1-55f4-42f9-932d-5c346785bdcf admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:55.525505 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1325/2609] 10.222.0.66 () {58 vars in 1152 bytes} [Mon Jul 18 02:42:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:42:55.525724 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83488df1-55f4-42f9-932d-5c346785bdcf admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:55.536880 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-e44b8b10-24b1-41fc-96b5-0b8d7e66f4ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d33beaa0fa641e58c501e7311caf194', 'request_id': 'req-e44b8b10-24b1-41fc-96b5-0b8d7e66f4ff'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:55.538937 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83488df1-55f4-42f9-932d-5c346785bdcf admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:55.544713 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-e44b8b10-24b1-41fc-96b5-0b8d7e66f4ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d33beaa0fa641e58c501e7311caf194', 'request_id': 'req-e44b8b10-24b1-41fc-96b5-0b8d7e66f4ff'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:55.553578 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-e44b8b10-24b1-41fc-96b5-0b8d7e66f4ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d33beaa0fa641e58c501e7311caf194', 'request_id': 'req-e44b8b10-24b1-41fc-96b5-0b8d7e66f4ff'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:55.635692 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-e44b8b10-24b1-41fc-96b5-0b8d7e66f4ff admin admin] The token cache is being invalidate because project 7d33beaa0fa641e58c501e7311caf194 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 02:42:55.636502 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-e44b8b10-24b1-41fc-96b5-0b8d7e66f4ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:55.636659 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-e44b8b10-24b1-41fc-96b5-0b8d7e66f4ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:55.639536 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1285/2610] 10.222.0.66 () {58 vars in 1232 bytes} [Mon Jul 18 02:42:55 2022] DELETE /identity/v3/projects/7d33beaa0fa641e58c501e7311caf194 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:42:55.640328 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e3b9a6-ddc5-4818-9dba-04e50e9b54bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:55.640474 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44b8b10-24b1-41fc-96b5-0b8d7e66f4ff admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:55.641902 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-e44b8b10-24b1-41fc-96b5-0b8d7e66f4ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:55.642181 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e3b9a6-ddc5-4818-9dba-04e50e9b54bf admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:55.642639 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e3b9a6-ddc5-4818-9dba-04e50e9b54bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:55.643581 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e3b9a6-ddc5-4818-9dba-04e50e9b54bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:55.644692 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e3b9a6-ddc5-4818-9dba-04e50e9b54bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:55.647064 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e3b9a6-ddc5-4818-9dba-04e50e9b54bf admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:55.657349 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44b8b10-24b1-41fc-96b5-0b8d7e66f4ff admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:55.662578 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0352442-dbf1-4974-9a75-2a861729a486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:55.663051 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1326/2611] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:42:55 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:42:55.663409 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0352442-dbf1-4974-9a75-2a861729a486 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:55.663933 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0352442-dbf1-4974-9a75-2a861729a486 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:55.664203 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0352442-dbf1-4974-9a75-2a861729a486 admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:55.664901 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0352442-dbf1-4974-9a75-2a861729a486 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:55.665350 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f821016-095a-4218-b681-b708e27b2823 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:55.666743 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f821016-095a-4218-b681-b708e27b2823 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:55.667190 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0352442-dbf1-4974-9a75-2a861729a486 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:55.667331 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f821016-095a-4218-b681-b708e27b2823 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:55.699914 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-9f821016-095a-4218-b681-b708e27b2823 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:42:55.702071 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-9f821016-095a-4218-b681-b708e27b2823 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:55.712578 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1327/2612] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:42:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:42:55.712578 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f821016-095a-4218-b681-b708e27b2823 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:55.718133 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-9f821016-095a-4218-b681-b708e27b2823 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:55.735294 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f821016-095a-4218-b681-b708e27b2823 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:55.744670 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313c3718-1dcd-432f-a7bb-6db61ba843bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:55.744670 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313c3718-1dcd-432f-a7bb-6db61ba843bc admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:55.744670 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313c3718-1dcd-432f-a7bb-6db61ba843bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:55.744670 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313c3718-1dcd-432f-a7bb-6db61ba843bc admin admin] PATH_INFO: `/v3/projects/8505b1f5965f4d04b1897f01c135d6ea/users/cb4702f0216b47b8ba83872eeee40173/roles/d4aa3324f0c84887b41d0dc4acf7f30e` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:55.763862 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313c3718-1dcd-432f-a7bb-6db61ba843bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=8505b1f5965f4d04b1897f01c135d6ea, user_id=cb4702f0216b47b8ba83872eeee40173, role_id=d4aa3324f0c84887b41d0dc4acf7f30e)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:55.767181 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313c3718-1dcd-432f-a7bb-6db61ba843bc admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:55.775336 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1286/2613] 10.222.0.66 () {60 vars in 1152 bytes} [Mon Jul 18 02:42:55 2022] POST /identity/v3/projects => generated 375 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:42:55.778188 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d17e5e4-616c-492b-a4f0-fc980982c8f4 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:55.778188 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d17e5e4-616c-492b-a4f0-fc980982c8f4 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:55.778188 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d17e5e4-616c-492b-a4f0-fc980982c8f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:55.799244 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-1d17e5e4-616c-492b-a4f0-fc980982c8f4 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:42:55.800483 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-1d17e5e4-616c-492b-a4f0-fc980982c8f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:55.868093 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1287/2614] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:42:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:42:55.874131 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1328/2615] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:42:55 2022] PUT /identity/v3/projects/8505b1f5965f4d04b1897f01c135d6ea/users/cb4702f0216b47b8ba83872eeee40173/roles/d4aa3324f0c84887b41d0dc4acf7f30e => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:42:55.874131 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313c3718-1dcd-432f-a7bb-6db61ba843bc admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:55.874368 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d17e5e4-616c-492b-a4f0-fc980982c8f4 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:55.874499 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-313c3718-1dcd-432f-a7bb-6db61ba843bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:55.884721 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d17e5e4-616c-492b-a4f0-fc980982c8f4 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:55.948695 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313c3718-1dcd-432f-a7bb-6db61ba843bc admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:55.952759 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb179b1e-fe5b-4a65-a2af-a39ffc04b124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:55.953045 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe72c530-5175-4c20-92a0-0882a96acbe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:55.953574 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb179b1e-fe5b-4a65-a2af-a39ffc04b124 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:55.953952 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb179b1e-fe5b-4a65-a2af-a39ffc04b124 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:55.954074 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe72c530-5175-4c20-92a0-0882a96acbe0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:55.954401 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe72c530-5175-4c20-92a0-0882a96acbe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:55.954504 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb179b1e-fe5b-4a65-a2af-a39ffc04b124 admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:55.954681 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe72c530-5175-4c20-92a0-0882a96acbe0 admin admin] PATH_INFO: `/v3/projects/fb9b0a66f65c4d42ad835d65ea5208e2` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:55.955380 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb179b1e-fe5b-4a65-a2af-a39ffc04b124 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:55.956253 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe72c530-5175-4c20-92a0-0882a96acbe0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb9b0a66f65c4d42ad835d65ea5208e2)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:55.957707 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb179b1e-fe5b-4a65-a2af-a39ffc04b124 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:55.958248 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe72c530-5175-4c20-92a0-0882a96acbe0 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:55.967830 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:42:56.101850 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1288/2616] 10.222.0.66 () {60 vars in 1143 bytes} [Mon Jul 18 02:42:55 2022] POST /identity/v3/users => generated 288 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:42:56.101850 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb179b1e-fe5b-4a65-a2af-a39ffc04b124 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:56.109893 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb179b1e-fe5b-4a65-a2af-a39ffc04b124 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:56.115210 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee51bc28-60b6-43b3-96dc-466fd42ecc78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:56.115871 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee51bc28-60b6-43b3-96dc-466fd42ecc78 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:56.116157 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee51bc28-60b6-43b3-96dc-466fd42ecc78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:56.116437 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee51bc28-60b6-43b3-96dc-466fd42ecc78 admin admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:56.117169 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee51bc28-60b6-43b3-96dc-466fd42ecc78 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:42:56.117485 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee51bc28-60b6-43b3-96dc-466fd42ecc78 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:56.119247 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee51bc28-60b6-43b3-96dc-466fd42ecc78 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:56.132678 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1289/2617] 10.222.0.66 () {58 vars in 1152 bytes} [Mon Jul 18 02:42:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:42:56.133738 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee51bc28-60b6-43b3-96dc-466fd42ecc78 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:56.144985 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee51bc28-60b6-43b3-96dc-466fd42ecc78 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:56.145925 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-fe72c530-5175-4c20-92a0-0882a96acbe0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb9b0a66f65c4d42ad835d65ea5208e2', 'request_id': 'req-fe72c530-5175-4c20-92a0-0882a96acbe0'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:56.151751 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-fe72c530-5175-4c20-92a0-0882a96acbe0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb9b0a66f65c4d42ad835d65ea5208e2', 'request_id': 'req-fe72c530-5175-4c20-92a0-0882a96acbe0'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:56.203953 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b46ad29-26ff-4db4-b14d-011c6c591b55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:56.204842 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b46ad29-26ff-4db4-b14d-011c6c591b55 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:56.205082 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b46ad29-26ff-4db4-b14d-011c6c591b55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:56.205774 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b46ad29-26ff-4db4-b14d-011c6c591b55 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:56.206534 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b46ad29-26ff-4db4-b14d-011c6c591b55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:56.207895 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b46ad29-26ff-4db4-b14d-011c6c591b55 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:56.221759 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1290/2618] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:42:56 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:42:56.221759 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b46ad29-26ff-4db4-b14d-011c6c591b55 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:56.229016 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-fe72c530-5175-4c20-92a0-0882a96acbe0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb9b0a66f65c4d42ad835d65ea5208e2', 'request_id': 'req-fe72c530-5175-4c20-92a0-0882a96acbe0'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:56.229720 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b46ad29-26ff-4db4-b14d-011c6c591b55 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:56.234746 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93a164d-b819-4808-847b-c4252d829d32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:56.239143 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93a164d-b819-4808-847b-c4252d829d32 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:56.239143 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93a164d-b819-4808-847b-c4252d829d32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:56.239143 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93a164d-b819-4808-847b-c4252d829d32 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:56.239143 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93a164d-b819-4808-847b-c4252d829d32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:56.239143 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93a164d-b819-4808-847b-c4252d829d32 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:56.242482 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-fe72c530-5175-4c20-92a0-0882a96acbe0 admin admin] The token cache is being invalidate because project fb9b0a66f65c4d42ad835d65ea5208e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 02:42:56.242832 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-fe72c530-5175-4c20-92a0-0882a96acbe0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:56.243554 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-fe72c530-5175-4c20-92a0-0882a96acbe0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:56.245735 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1329/2619] 10.222.0.66 () {58 vars in 1232 bytes} [Mon Jul 18 02:42:55 2022] DELETE /identity/v3/projects/fb9b0a66f65c4d42ad835d65ea5208e2 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:42:56.246541 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe72c530-5175-4c20-92a0-0882a96acbe0 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:56.249739 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-fe72c530-5175-4c20-92a0-0882a96acbe0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:56.253103 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1291/2620] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:42:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:42:56.253103 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93a164d-b819-4808-847b-c4252d829d32 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:56.254425 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-f93a164d-b819-4808-847b-c4252d829d32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:56.264694 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe72c530-5175-4c20-92a0-0882a96acbe0 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:56.269912 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33eb827e-74f1-4bbd-830a-53973faf7436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:56.270188 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93a164d-b819-4808-847b-c4252d829d32 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:56.270615 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eb827e-74f1-4bbd-830a-53973faf7436 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:56.270903 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eb827e-74f1-4bbd-830a-53973faf7436 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:56.271343 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eb827e-74f1-4bbd-830a-53973faf7436 admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:56.272141 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33eb827e-74f1-4bbd-830a-53973faf7436 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:56.274076 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33eb827e-74f1-4bbd-830a-53973faf7436 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:56.275235 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e5506a-e895-40cc-ab38-226d95809a49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:56.276138 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e5506a-e895-40cc-ab38-226d95809a49 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:56.276426 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e5506a-e895-40cc-ab38-226d95809a49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:56.276704 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e5506a-e895-40cc-ab38-226d95809a49 admin admin] PATH_INFO: `/v3/projects/8505b1f5965f4d04b1897f01c135d6ea/users/cb4702f0216b47b8ba83872eeee40173/roles/d4aa3324f0c84887b41d0dc4acf7f30e` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:56.280411 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e5506a-e895-40cc-ab38-226d95809a49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8505b1f5965f4d04b1897f01c135d6ea, user_id=cb4702f0216b47b8ba83872eeee40173, role_id=d4aa3324f0c84887b41d0dc4acf7f30e)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:56.283519 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e5506a-e895-40cc-ab38-226d95809a49 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:56.298669 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1292/2621] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:42:56 2022] PUT /identity/v3/projects/8505b1f5965f4d04b1897f01c135d6ea/users/cb4702f0216b47b8ba83872eeee40173/roles/d4aa3324f0c84887b41d0dc4acf7f30e => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:42:56.303158 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e5506a-e895-40cc-ab38-226d95809a49 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:56.304882 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-82e5506a-e895-40cc-ab38-226d95809a49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:56.373999 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e5506a-e895-40cc-ab38-226d95809a49 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:56.375267 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1330/2622] 10.222.0.66 () {60 vars in 1152 bytes} [Mon Jul 18 02:42:56 2022] POST /identity/v3/projects => generated 385 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:42:56.376186 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33eb827e-74f1-4bbd-830a-53973faf7436 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:56.377844 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-33eb827e-74f1-4bbd-830a-53973faf7436 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:56.432213 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8db325f-6ea4-471b-b825-b1ff340a6797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:56.433113 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8db325f-6ea4-471b-b825-b1ff340a6797 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:56.433409 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8db325f-6ea4-471b-b825-b1ff340a6797 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:56.433675 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8db325f-6ea4-471b-b825-b1ff340a6797 admin admin] PATH_INFO: `/v3/users/4bded16043ca4b7cb1c96bdbb94a741d` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:56.435551 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8db325f-6ea4-471b-b825-b1ff340a6797 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4bded16043ca4b7cb1c96bdbb94a741d)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:56.438234 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8db325f-6ea4-471b-b825-b1ff340a6797 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:56.460652 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33eb827e-74f1-4bbd-830a-53973faf7436 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:56.466284 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a65f4b-b5f1-42f4-a36d-0b7d2d1deb5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:56.468270 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a65f4b-b5f1-42f4-a36d-0b7d2d1deb5f admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:56.468270 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a65f4b-b5f1-42f4-a36d-0b7d2d1deb5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:56.469115 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a65f4b-b5f1-42f4-a36d-0b7d2d1deb5f admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:56.469486 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a65f4b-b5f1-42f4-a36d-0b7d2d1deb5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:56.471135 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a65f4b-b5f1-42f4-a36d-0b7d2d1deb5f admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:56.486228 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1331/2623] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:42:56 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:42:56.487175 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a65f4b-b5f1-42f4-a36d-0b7d2d1deb5f admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:56.502449 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a65f4b-b5f1-42f4-a36d-0b7d2d1deb5f admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:56.508836 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2b804f-986b-42cd-9105-e4885fe97978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:56.509555 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2b804f-986b-42cd-9105-e4885fe97978 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:56.509845 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2b804f-986b-42cd-9105-e4885fe97978 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:56.510198 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2b804f-986b-42cd-9105-e4885fe97978 admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:56.511737 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2b804f-986b-42cd-9105-e4885fe97978 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:56.516884 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2b804f-986b-42cd-9105-e4885fe97978 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:56.621592 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1332/2624] 10.222.0.66 () {60 vars in 1143 bytes} [Mon Jul 18 02:42:56 2022] POST /identity/v3/users => generated 293 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:42:56.621592 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2b804f-986b-42cd-9105-e4885fe97978 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:56.631086 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2b804f-986b-42cd-9105-e4885fe97978 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:56.636075 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f67fc9-b573-4159-915f-39e942b5cce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:56.636971 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f67fc9-b573-4159-915f-39e942b5cce3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:56.637257 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f67fc9-b573-4159-915f-39e942b5cce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:56.637534 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f67fc9-b573-4159-915f-39e942b5cce3 admin admin] PATH_INFO: `/v3/projects/e8200538fea84e46b424ede0d9b719d8/users/2715041954204ec4bd55d5b5719e9bd8/roles/65feabf3b27944a8b01a86c8ac608457` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:56.654399 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f67fc9-b573-4159-915f-39e942b5cce3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8200538fea84e46b424ede0d9b719d8, user_id=2715041954204ec4bd55d5b5719e9bd8, role_id=65feabf3b27944a8b01a86c8ac608457)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:56.657661 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f67fc9-b573-4159-915f-39e942b5cce3 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:56.674994 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-e8db325f-6ea4-471b-b825-b1ff340a6797 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4bded16043ca4b7cb1c96bdbb94a741d'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:56.830841 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1333/2625] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:42:56 2022] PUT /identity/v3/projects/e8200538fea84e46b424ede0d9b719d8/users/2715041954204ec4bd55d5b5719e9bd8/roles/65feabf3b27944a8b01a86c8ac608457 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:42:56.831756 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f67fc9-b573-4159-915f-39e942b5cce3 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:56.910289 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f67fc9-b573-4159-915f-39e942b5cce3 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:56.915217 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64a6b6f-d994-4c51-ad02-ee5894812d2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:56.915899 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64a6b6f-d994-4c51-ad02-ee5894812d2b admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:56.916192 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64a6b6f-d994-4c51-ad02-ee5894812d2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:56.916475 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64a6b6f-d994-4c51-ad02-ee5894812d2b admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:56.917231 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64a6b6f-d994-4c51-ad02-ee5894812d2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:56.919033 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64a6b6f-d994-4c51-ad02-ee5894812d2b admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:56.930910 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1334/2626] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:42:56 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:42:56.932674 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577bed35-a557-454a-b9be-688566824858 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:56.932971 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577bed35-a557-454a-b9be-688566824858 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:56.933254 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577bed35-a557-454a-b9be-688566824858 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:56.939619 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-e8db325f-6ea4-471b-b825-b1ff340a6797 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4bded16043ca4b7cb1c96bdbb94a741d'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:56.941731 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-e8db325f-6ea4-471b-b825-b1ff340a6797 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4bded16043ca4b7cb1c96bdbb94a741d'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:56.962444 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-e8db325f-6ea4-471b-b825-b1ff340a6797 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4bded16043ca4b7cb1c96bdbb94a741d'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:56.963498 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-e8db325f-6ea4-471b-b825-b1ff340a6797 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4bded16043ca4b7cb1c96bdbb94a741d'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:56.971303 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-577bed35-a557-454a-b9be-688566824858 None None] MFA Rules not processed for user `2715041954204ec4bd55d5b5719e9bd8`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:42:56.973359 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-577bed35-a557-454a-b9be-688566824858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:56.979599 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1293/2627] 10.222.0.66 () {58 vars in 1223 bytes} [Mon Jul 18 02:42:56 2022] DELETE /identity/v3/users/4bded16043ca4b7cb1c96bdbb94a741d => generated 0 bytes in 677 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:42:56.980596 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8db325f-6ea4-471b-b825-b1ff340a6797 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:56.982337 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-e8db325f-6ea4-471b-b825-b1ff340a6797 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:57.035423 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:42:57.070917 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1335/2628] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:42:56 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:42:57.077666 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577bed35-a557-454a-b9be-688566824858 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:57.083257 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-577bed35-a557-454a-b9be-688566824858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:57.100634 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8db325f-6ea4-471b-b825-b1ff340a6797 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:57.103270 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577bed35-a557-454a-b9be-688566824858 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:57.109137 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9edbff-8b71-4aeb-95cc-08fc23348d0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:57.110894 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9edbff-8b71-4aeb-95cc-08fc23348d0e admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:57.111143 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df1fb5f-e8e3-497c-82b2-75cb1051160a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:57.111538 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9edbff-8b71-4aeb-95cc-08fc23348d0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:57.111970 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9edbff-8b71-4aeb-95cc-08fc23348d0e admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:57.113117 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9edbff-8b71-4aeb-95cc-08fc23348d0e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:57.113532 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df1fb5f-e8e3-497c-82b2-75cb1051160a admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:57.114439 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df1fb5f-e8e3-497c-82b2-75cb1051160a admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:57.114798 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df1fb5f-e8e3-497c-82b2-75cb1051160a admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:57.115082 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9edbff-8b71-4aeb-95cc-08fc23348d0e admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:57.116758 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df1fb5f-e8e3-497c-82b2-75cb1051160a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:57.118706 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df1fb5f-e8e3-497c-82b2-75cb1051160a admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:57.131984 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1294/2629] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:42:56 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:42:57.136053 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df1fb5f-e8e3-497c-82b2-75cb1051160a admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:57.146731 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df1fb5f-e8e3-497c-82b2-75cb1051160a admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:57.151819 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3975314-ffb6-493a-82bf-0c55d32b3aee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:57.152897 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3975314-ffb6-493a-82bf-0c55d32b3aee admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:57.153398 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3975314-ffb6-493a-82bf-0c55d32b3aee admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:57.153578 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3975314-ffb6-493a-82bf-0c55d32b3aee admin admin] PATH_INFO: `/v3/projects/f5484a21aa1b4b378fa3e6e5f4fec572/users/32288d7e54cc49ea9260c4fbfa2e25a4/roles/65feabf3b27944a8b01a86c8ac608457` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:57.169586 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3975314-ffb6-493a-82bf-0c55d32b3aee admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5484a21aa1b4b378fa3e6e5f4fec572, user_id=32288d7e54cc49ea9260c4fbfa2e25a4, role_id=65feabf3b27944a8b01a86c8ac608457)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:57.172710 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3975314-ffb6-493a-82bf-0c55d32b3aee admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:57.220277 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1336/2630] 10.222.0.66 () {60 vars in 1152 bytes} [Mon Jul 18 02:42:57 2022] POST /identity/v3/projects => generated 377 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:42:57.221471 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9edbff-8b71-4aeb-95cc-08fc23348d0e admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:57.256479 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1295/2631] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:42:57 2022] PUT /identity/v3/projects/f5484a21aa1b4b378fa3e6e5f4fec572/users/32288d7e54cc49ea9260c4fbfa2e25a4/roles/65feabf3b27944a8b01a86c8ac608457 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:42:57.269930 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc46b8a8-3d97-41af-9a29-1d401a517485 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:57.270550 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc46b8a8-3d97-41af-9a29-1d401a517485 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:57.271000 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc46b8a8-3d97-41af-9a29-1d401a517485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:57.305160 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-dc46b8a8-3d97-41af-9a29-1d401a517485 None None] MFA Rules not processed for user `32288d7e54cc49ea9260c4fbfa2e25a4`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:42:57.306424 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-dc46b8a8-3d97-41af-9a29-1d401a517485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:57.328950 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9edbff-8b71-4aeb-95cc-08fc23348d0e admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:57.462270 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1296/2632] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:42:57 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 02:42:57.467936 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627a712f-386e-4466-8111-50fb0d3773df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:57.468784 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627a712f-386e-4466-8111-50fb0d3773df admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:57.469073 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627a712f-386e-4466-8111-50fb0d3773df admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:57.469430 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627a712f-386e-4466-8111-50fb0d3773df admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:57.470190 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627a712f-386e-4466-8111-50fb0d3773df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:57.478730 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627a712f-386e-4466-8111-50fb0d3773df admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:57.480948 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc46b8a8-3d97-41af-9a29-1d401a517485 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:57.483741 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-dc46b8a8-3d97-41af-9a29-1d401a517485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:57.574383 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1337/2633] 10.222.0.66 () {60 vars in 1143 bytes} [Mon Jul 18 02:42:57 2022] POST /identity/v3/users => generated 295 bytes in 353 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:42:57.577637 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc46b8a8-3d97-41af-9a29-1d401a517485 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:57.579980 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627a712f-386e-4466-8111-50fb0d3773df admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:57.581848 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89768de9-0360-46df-aec1-7241e2503be4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd49388facdf446885e55b45ef91f01e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:57.595776 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89768de9-0360-46df-aec1-7241e2503be4 service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:57.602550 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89768de9-0360-46df-aec1-7241e2503be4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:57.602876 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89768de9-0360-46df-aec1-7241e2503be4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:57.622913 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89768de9-0360-46df-aec1-7241e2503be4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:57.624773 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89768de9-0360-46df-aec1-7241e2503be4 service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:57.630957 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627a712f-386e-4466-8111-50fb0d3773df admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:57.641703 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525ea2f7-654a-475c-bdbb-5529e7254944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:57.641703 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525ea2f7-654a-475c-bdbb-5529e7254944 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:57.641703 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525ea2f7-654a-475c-bdbb-5529e7254944 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:57.643367 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525ea2f7-654a-475c-bdbb-5529e7254944 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:57.644486 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525ea2f7-654a-475c-bdbb-5529e7254944 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:57.645790 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1297/2634] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:42:57 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:42:57.646307 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525ea2f7-654a-475c-bdbb-5529e7254944 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:57.647212 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89768de9-0360-46df-aec1-7241e2503be4 service cinder] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:57.650340 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-89768de9-0360-46df-aec1-7241e2503be4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:57.661968 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1338/2635] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:42:57 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:42:57.666154 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525ea2f7-654a-475c-bdbb-5529e7254944 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:57.670475 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89768de9-0360-46df-aec1-7241e2503be4 service cinder] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:57.678636 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683e95fb-ae41-4519-ba79-d697ccd841ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:57.679621 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683e95fb-ae41-4519-ba79-d697ccd841ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:57.679974 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683e95fb-ae41-4519-ba79-d697ccd841ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:57.680320 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683e95fb-ae41-4519-ba79-d697ccd841ac admin admin] PATH_INFO: `/v3/projects/4090f7541d8042f4bdd8e2af4bc1f52d` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:57.680672 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525ea2f7-654a-475c-bdbb-5529e7254944 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:57.681722 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683e95fb-ae41-4519-ba79-d697ccd841ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=4090f7541d8042f4bdd8e2af4bc1f52d)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:57.684188 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683e95fb-ae41-4519-ba79-d697ccd841ac admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:57.686573 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c1df6d-d2a1-46d8-8bff-47adcef5113a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:57.687798 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c1df6d-d2a1-46d8-8bff-47adcef5113a admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:57.688084 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c1df6d-d2a1-46d8-8bff-47adcef5113a admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:57.688363 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c1df6d-d2a1-46d8-8bff-47adcef5113a admin admin] PATH_INFO: `/v3/projects/05b150a2fea54fd8895ae364690e28e0/users/9b4c027b17bf42e4a6b49b9394fd64dc/roles/d4aa3324f0c84887b41d0dc4acf7f30e` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:57.707492 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c1df6d-d2a1-46d8-8bff-47adcef5113a admin admin] RBAC: Authorizing `identity:create_grant(project_id=05b150a2fea54fd8895ae364690e28e0, user_id=9b4c027b17bf42e4a6b49b9394fd64dc, role_id=d4aa3324f0c84887b41d0dc4acf7f30e)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:57.710309 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c1df6d-d2a1-46d8-8bff-47adcef5113a admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:57.793483 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1339/2636] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:42:57 2022] PUT /identity/v3/projects/05b150a2fea54fd8895ae364690e28e0/users/9b4c027b17bf42e4a6b49b9394fd64dc/roles/d4aa3324f0c84887b41d0dc4acf7f30e => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:42:57.795579 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c1df6d-d2a1-46d8-8bff-47adcef5113a admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:57.843031 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-683e95fb-ae41-4519-ba79-d697ccd841ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4090f7541d8042f4bdd8e2af4bc1f52d', 'request_id': 'req-683e95fb-ae41-4519-ba79-d697ccd841ac'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:57.869621 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-683e95fb-ae41-4519-ba79-d697ccd841ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4090f7541d8042f4bdd8e2af4bc1f52d', 'request_id': 'req-683e95fb-ae41-4519-ba79-d697ccd841ac'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:57.880792 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-683e95fb-ae41-4519-ba79-d697ccd841ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4090f7541d8042f4bdd8e2af4bc1f52d', 'request_id': 'req-683e95fb-ae41-4519-ba79-d697ccd841ac'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:57.898374 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c1df6d-d2a1-46d8-8bff-47adcef5113a admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:57.973602 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9804d490-813c-4c1c-9dfe-9af86024d825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:57.974405 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9804d490-813c-4c1c-9dfe-9af86024d825 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:57.974753 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9804d490-813c-4c1c-9dfe-9af86024d825 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:57.975119 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9804d490-813c-4c1c-9dfe-9af86024d825 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:57.975910 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9804d490-813c-4c1c-9dfe-9af86024d825 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:57.977819 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9804d490-813c-4c1c-9dfe-9af86024d825 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:57.990142 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1340/2637] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:42:57 2022] GET /identity/v3/roles => generated 1420 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:42:57.992985 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9804d490-813c-4c1c-9dfe-9af86024d825 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:58.010528 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-683e95fb-ae41-4519-ba79-d697ccd841ac admin admin] The token cache is being invalidate because project 4090f7541d8042f4bdd8e2af4bc1f52d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 02:42:58.023729 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-683e95fb-ae41-4519-ba79-d697ccd841ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:58.023729 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-683e95fb-ae41-4519-ba79-d697ccd841ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:42:58.023729 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1298/2638] 10.222.0.66 () {58 vars in 1232 bytes} [Mon Jul 18 02:42:57 2022] DELETE /identity/v3/projects/4090f7541d8042f4bdd8e2af4bc1f52d => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:42:58.024286 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9804d490-813c-4c1c-9dfe-9af86024d825 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:58.026694 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655fe415-2871-4f7c-ae66-5e337e1b3a3e None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:58.027176 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655fe415-2871-4f7c-ae66-5e337e1b3a3e None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:58.027628 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655fe415-2871-4f7c-ae66-5e337e1b3a3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:58.028281 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd37a4bd-72fb-4210-8593-c31a388b04fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:58.029462 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd37a4bd-72fb-4210-8593-c31a388b04fe admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:58.029768 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd37a4bd-72fb-4210-8593-c31a388b04fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:58.030029 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd37a4bd-72fb-4210-8593-c31a388b04fe admin admin] PATH_INFO: `/v3/projects/05b150a2fea54fd8895ae364690e28e0/users/9b4c027b17bf42e4a6b49b9394fd64dc/roles/d4aa3324f0c84887b41d0dc4acf7f30e` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:58.032507 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd37a4bd-72fb-4210-8593-c31a388b04fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=05b150a2fea54fd8895ae364690e28e0, user_id=9b4c027b17bf42e4a6b49b9394fd64dc, role_id=d4aa3324f0c84887b41d0dc4acf7f30e)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:58.035471 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd37a4bd-72fb-4210-8593-c31a388b04fe admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:58.054565 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-655fe415-2871-4f7c-ae66-5e337e1b3a3e None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:42:58.055959 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-655fe415-2871-4f7c-ae66-5e337e1b3a3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:58.057783 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1341/2639] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:42:57 2022] PUT /identity/v3/projects/05b150a2fea54fd8895ae364690e28e0/users/9b4c027b17bf42e4a6b49b9394fd64dc/roles/d4aa3324f0c84887b41d0dc4acf7f30e => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 02:42:58.064867 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c48b34c-8ad3-4246-b215-d0afe2c58639 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:58.064867 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c48b34c-8ad3-4246-b215-d0afe2c58639 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:58.064867 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c48b34c-8ad3-4246-b215-d0afe2c58639 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:58.109678 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-0c48b34c-8ad3-4246-b215-d0afe2c58639 None None] MFA Rules not processed for user `9b4c027b17bf42e4a6b49b9394fd64dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:42:58.111257 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-0c48b34c-8ad3-4246-b215-d0afe2c58639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:58.142181 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:42:58.157518 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1299/2640] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:42:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:42:58.172967 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b8a94d-67ba-43ec-9226-85a4e7ed2d8a None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:58.174018 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b8a94d-67ba-43ec-9226-85a4e7ed2d8a None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:58.174634 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b8a94d-67ba-43ec-9226-85a4e7ed2d8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:58.209733 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-d2b8a94d-67ba-43ec-9226-85a4e7ed2d8a None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:42:58.210402 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1342/2641] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:42:58 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:42:58.211870 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-d2b8a94d-67ba-43ec-9226-85a4e7ed2d8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:58.224113 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c48b34c-8ad3-4246-b215-d0afe2c58639 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:58.227797 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1300/2642] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:42:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:42:58.232217 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b8a94d-67ba-43ec-9226-85a4e7ed2d8a None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:58.233104 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-0c48b34c-8ad3-4246-b215-d0afe2c58639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:58.245094 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b8a94d-67ba-43ec-9226-85a4e7ed2d8a None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:58.252539 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2723918e-c142-4399-961c-d2a46756a262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:58.253591 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2723918e-c142-4399-961c-d2a46756a262 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:58.253945 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2723918e-c142-4399-961c-d2a46756a262 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:58.254505 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2723918e-c142-4399-961c-d2a46756a262 admin admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:58.255391 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2723918e-c142-4399-961c-d2a46756a262 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:42:58.256028 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2723918e-c142-4399-961c-d2a46756a262 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:58.257870 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2723918e-c142-4399-961c-d2a46756a262 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:58.271744 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1301/2643] 10.222.0.66 () {58 vars in 1152 bytes} [Mon Jul 18 02:42:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:42:58.276543 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2723918e-c142-4399-961c-d2a46756a262 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:58.290878 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2723918e-c142-4399-961c-d2a46756a262 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:58.298353 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686a3231-ec14-4cd7-a88e-f9eb63bdcfa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:58.299397 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686a3231-ec14-4cd7-a88e-f9eb63bdcfa5 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:58.299865 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686a3231-ec14-4cd7-a88e-f9eb63bdcfa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:58.300445 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686a3231-ec14-4cd7-a88e-f9eb63bdcfa5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:58.301348 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686a3231-ec14-4cd7-a88e-f9eb63bdcfa5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:58.303670 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686a3231-ec14-4cd7-a88e-f9eb63bdcfa5 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:58.346294 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c48b34c-8ad3-4246-b215-d0afe2c58639 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:58.351447 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffd44de-9688-4355-aa1a-302e6741b93e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd49388facdf446885e55b45ef91f01e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:58.352326 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffd44de-9688-4355-aa1a-302e6741b93e service cinder] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:58.352618 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffd44de-9688-4355-aa1a-302e6741b93e service cinder] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:58.352900 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffd44de-9688-4355-aa1a-302e6741b93e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:58.355054 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-1ffd44de-9688-4355-aa1a-302e6741b93e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:58.430641 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffd44de-9688-4355-aa1a-302e6741b93e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:58.433299 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffd44de-9688-4355-aa1a-302e6741b93e service cinder] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:58.463149 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1302/2645] 10.222.0.66 () {60 vars in 1152 bytes} [Mon Jul 18 02:42:58 2022] POST /identity/v3/projects => generated 385 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 02:42:58.464067 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1343/2645] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:42:58 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 02:42:58.472153 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686a3231-ec14-4cd7-a88e-f9eb63bdcfa5 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:58.699553 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686a3231-ec14-4cd7-a88e-f9eb63bdcfa5 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:58.707268 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b588105d-f801-4351-9c26-85192076e367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:58.709271 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b588105d-f801-4351-9c26-85192076e367 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:58.710316 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b588105d-f801-4351-9c26-85192076e367 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:58.710763 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b588105d-f801-4351-9c26-85192076e367 admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:58.712468 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b588105d-f801-4351-9c26-85192076e367 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:58.714796 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b588105d-f801-4351-9c26-85192076e367 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:58.739375 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:42:58.886416 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1303/2646] 10.222.0.66 () {60 vars in 1143 bytes} [Mon Jul 18 02:42:58 2022] POST /identity/v3/users => generated 293 bytes in 419 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:42:58.891952 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffd44de-9688-4355-aa1a-302e6741b93e service cinder] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:58.925650 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffd44de-9688-4355-aa1a-302e6741b93e service cinder] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:58.930803 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ade683-66e3-402d-8f68-f5d61d881c86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:58.931565 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ade683-66e3-402d-8f68-f5d61d881c86 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:58.931927 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ade683-66e3-402d-8f68-f5d61d881c86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:58.932277 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ade683-66e3-402d-8f68-f5d61d881c86 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:58.933088 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ade683-66e3-402d-8f68-f5d61d881c86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:58.935774 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ade683-66e3-402d-8f68-f5d61d881c86 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:58.960874 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1344/2647] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:42:58 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:42:58.968186 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b588105d-f801-4351-9c26-85192076e367 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:58.981799 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b588105d-f801-4351-9c26-85192076e367 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:58.990444 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbc5a6b-8f13-4981-b4d9-02d52987c8b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:58.991167 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbc5a6b-8f13-4981-b4d9-02d52987c8b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:58.991456 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbc5a6b-8f13-4981-b4d9-02d52987c8b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:58.991814 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbc5a6b-8f13-4981-b4d9-02d52987c8b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:58.992746 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbc5a6b-8f13-4981-b4d9-02d52987c8b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:58.994825 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbc5a6b-8f13-4981-b4d9-02d52987c8b0 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:59.010141 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1304/2648] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:42:58 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:42:59.014415 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ade683-66e3-402d-8f68-f5d61d881c86 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:59.026215 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ade683-66e3-402d-8f68-f5d61d881c86 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:59.035806 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee74a32c-79be-4c7f-a55f-5bda30c40917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:59.036752 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee74a32c-79be-4c7f-a55f-5bda30c40917 admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:59.037043 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee74a32c-79be-4c7f-a55f-5bda30c40917 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:59.037383 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee74a32c-79be-4c7f-a55f-5bda30c40917 admin admin] PATH_INFO: `/v3/projects/5ef7150f06aa4cbdab1ca9407581a50d/users/a164da80f13446f0aa26527aeeb638ab/roles/65feabf3b27944a8b01a86c8ac608457` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:59.058004 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee74a32c-79be-4c7f-a55f-5bda30c40917 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ef7150f06aa4cbdab1ca9407581a50d, user_id=a164da80f13446f0aa26527aeeb638ab, role_id=65feabf3b27944a8b01a86c8ac608457)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:59.060791 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee74a32c-79be-4c7f-a55f-5bda30c40917 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:59.154940 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1345/2649] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:42:59 2022] PUT /identity/v3/projects/5ef7150f06aa4cbdab1ca9407581a50d/users/a164da80f13446f0aa26527aeeb638ab/roles/65feabf3b27944a8b01a86c8ac608457 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:42:59.158355 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f3363d-019f-4edd-9e89-6e9a6ae3a2fc None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:59.158659 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f3363d-019f-4edd-9e89-6e9a6ae3a2fc None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:59.158934 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f3363d-019f-4edd-9e89-6e9a6ae3a2fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:59.200325 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-99f3363d-019f-4edd-9e89-6e9a6ae3a2fc None None] MFA Rules not processed for user `a164da80f13446f0aa26527aeeb638ab`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:42:59.204053 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-99f3363d-019f-4edd-9e89-6e9a6ae3a2fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:42:59.275319 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1305/2650] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:42:59 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:42:59.280793 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee74a32c-79be-4c7f-a55f-5bda30c40917 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:59.365006 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee74a32c-79be-4c7f-a55f-5bda30c40917 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:59.371502 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d98eeac-0858-4004-998b-eb2b7eb50533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:59.373436 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d98eeac-0858-4004-998b-eb2b7eb50533 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:59.373788 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d98eeac-0858-4004-998b-eb2b7eb50533 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:59.375168 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d98eeac-0858-4004-998b-eb2b7eb50533 admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:59.375168 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d98eeac-0858-4004-998b-eb2b7eb50533 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:59.377564 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d98eeac-0858-4004-998b-eb2b7eb50533 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:59.493788 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1346/2651] 10.222.0.66 () {60 vars in 1152 bytes} [Mon Jul 18 02:42:59 2022] POST /identity/v3/projects => generated 385 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:42:59.498821 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f3363d-019f-4edd-9e89-6e9a6ae3a2fc None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:59.510200 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f3363d-019f-4edd-9e89-6e9a6ae3a2fc None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:59.612605 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7ec3c2-fc97-4553-bbfe-4a087b0960a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:59.612605 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7ec3c2-fc97-4553-bbfe-4a087b0960a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:59.612605 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7ec3c2-fc97-4553-bbfe-4a087b0960a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:59.612605 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7ec3c2-fc97-4553-bbfe-4a087b0960a0 admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:59.612972 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7ec3c2-fc97-4553-bbfe-4a087b0960a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:59.612972 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7ec3c2-fc97-4553-bbfe-4a087b0960a0 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:59.732112 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1306/2652] 10.222.0.66 () {60 vars in 1143 bytes} [Mon Jul 18 02:42:59 2022] POST /identity/v3/users => generated 299 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:42:59.736765 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d98eeac-0858-4004-998b-eb2b7eb50533 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:59.768087 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d98eeac-0858-4004-998b-eb2b7eb50533 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:59.777996 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edb59fa-408c-4179-9cd0-c5d68fffffbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:59.779088 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edb59fa-408c-4179-9cd0-c5d68fffffbd admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:59.779650 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edb59fa-408c-4179-9cd0-c5d68fffffbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:59.780272 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edb59fa-408c-4179-9cd0-c5d68fffffbd admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:59.781458 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edb59fa-408c-4179-9cd0-c5d68fffffbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:59.783383 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edb59fa-408c-4179-9cd0-c5d68fffffbd admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:59.806841 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1347/2653] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:42:59 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:42:59.813751 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7ec3c2-fc97-4553-bbfe-4a087b0960a0 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:59.824891 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7ec3c2-fc97-4553-bbfe-4a087b0960a0 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:42:59.832274 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557a0c98-6d41-498e-a2b0-7690984b83f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:42:59.833941 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557a0c98-6d41-498e-a2b0-7690984b83f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:42:59.834517 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557a0c98-6d41-498e-a2b0-7690984b83f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:42:59.837111 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557a0c98-6d41-498e-a2b0-7690984b83f7 admin admin] PATH_INFO: `/v3/projects/8eab6f0a2a46471f9aa267084e9e40fe/users/e4f5da7039104233b2d7d4c107b81140/roles/d4aa3324f0c84887b41d0dc4acf7f30e` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:42:59.857788 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557a0c98-6d41-498e-a2b0-7690984b83f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8eab6f0a2a46471f9aa267084e9e40fe, user_id=e4f5da7039104233b2d7d4c107b81140, role_id=d4aa3324f0c84887b41d0dc4acf7f30e)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:42:59.860759 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557a0c98-6d41-498e-a2b0-7690984b83f7 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:42:59.952307 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1307/2654] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:42:59 2022] PUT /identity/v3/projects/8eab6f0a2a46471f9aa267084e9e40fe/users/e4f5da7039104233b2d7d4c107b81140/roles/d4aa3324f0c84887b41d0dc4acf7f30e => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:42:59.956212 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edb59fa-408c-4179-9cd0-c5d68fffffbd admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:42:59.970836 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edb59fa-408c-4179-9cd0-c5d68fffffbd admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:43:00.025552 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:43:00.064167 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbe840e-3356-4df0-a6cc-240af8c1b043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:43:00.065117 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbe840e-3356-4df0-a6cc-240af8c1b043 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:00.067578 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbe840e-3356-4df0-a6cc-240af8c1b043 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:00.067578 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbe840e-3356-4df0-a6cc-240af8c1b043 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:00.067578 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbe840e-3356-4df0-a6cc-240af8c1b043 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:43:00.068424 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbe840e-3356-4df0-a6cc-240af8c1b043 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:43:00.080176 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1348/2655] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:42:59 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:43:00.084486 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557a0c98-6d41-498e-a2b0-7690984b83f7 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:43:00.095354 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557a0c98-6d41-498e-a2b0-7690984b83f7 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:43:00.100149 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9666bf-a31d-4440-a7a1-f6f83d7be93c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:43:00.101034 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9666bf-a31d-4440-a7a1-f6f83d7be93c admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:00.101403 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9666bf-a31d-4440-a7a1-f6f83d7be93c admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:00.101584 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9666bf-a31d-4440-a7a1-f6f83d7be93c admin admin] PATH_INFO: `/v3/projects/8eab6f0a2a46471f9aa267084e9e40fe/users/e4f5da7039104233b2d7d4c107b81140/roles/d4aa3324f0c84887b41d0dc4acf7f30e` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:00.103613 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9666bf-a31d-4440-a7a1-f6f83d7be93c admin admin] RBAC: Authorizing `identity:create_grant(project_id=8eab6f0a2a46471f9aa267084e9e40fe, user_id=e4f5da7039104233b2d7d4c107b81140, role_id=d4aa3324f0c84887b41d0dc4acf7f30e)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:43:00.106492 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9666bf-a31d-4440-a7a1-f6f83d7be93c admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:43:00.117649 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1308/2656] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:43:00 2022] PUT /identity/v3/projects/8eab6f0a2a46471f9aa267084e9e40fe/users/e4f5da7039104233b2d7d4c107b81140/roles/d4aa3324f0c84887b41d0dc4acf7f30e => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 02:43:00.130495 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92534bf-160c-487a-8db7-e7bae337c144 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:00.130495 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92534bf-160c-487a-8db7-e7bae337c144 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:00.130495 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92534bf-160c-487a-8db7-e7bae337c144 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:00.172564 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-e92534bf-160c-487a-8db7-e7bae337c144 None None] MFA Rules not processed for user `e4f5da7039104233b2d7d4c107b81140`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:43:00.174510 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-e92534bf-160c-487a-8db7-e7bae337c144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:43:00.280884 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1349/2657] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:43:00 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:43:00.291737 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9666bf-a31d-4440-a7a1-f6f83d7be93c admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:43:00.293757 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-bf9666bf-a31d-4440-a7a1-f6f83d7be93c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:43:00.411638 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9666bf-a31d-4440-a7a1-f6f83d7be93c admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:43:00.415314 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf7c93c-4092-4d1a-9514-0fc7a36466b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd49388facdf446885e55b45ef91f01e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:43:00.415967 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf7c93c-4092-4d1a-9514-0fc7a36466b4 service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:00.416479 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf7c93c-4092-4d1a-9514-0fc7a36466b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:00.416923 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf7c93c-4092-4d1a-9514-0fc7a36466b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:00.489780 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf7c93c-4092-4d1a-9514-0fc7a36466b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:43:00.491205 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf7c93c-4092-4d1a-9514-0fc7a36466b4 service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:43:00.502118 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1309/2658] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:43:00 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:43:00.754001 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:43:01.119909 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92534bf-160c-487a-8db7-e7bae337c144 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:43:01.122127 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-e92534bf-160c-487a-8db7-e7bae337c144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:43:01.214206 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92534bf-160c-487a-8db7-e7bae337c144 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:43:01.221190 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8017dc93-4f68-4b2e-9cfc-9513c97edabf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:43:01.222243 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8017dc93-4f68-4b2e-9cfc-9513c97edabf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:01.222705 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8017dc93-4f68-4b2e-9cfc-9513c97edabf admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:01.222986 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8017dc93-4f68-4b2e-9cfc-9513c97edabf admin admin] PATH_INFO: `/v3/users/32288d7e54cc49ea9260c4fbfa2e25a4` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:01.224266 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8017dc93-4f68-4b2e-9cfc-9513c97edabf admin admin] RBAC: Authorizing `identity:delete_user(user_id=32288d7e54cc49ea9260c4fbfa2e25a4)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:43:01.226444 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8017dc93-4f68-4b2e-9cfc-9513c97edabf admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:43:01.453401 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-8017dc93-4f68-4b2e-9cfc-9513c97edabf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32288d7e54cc49ea9260c4fbfa2e25a4'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:01.513694 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-8017dc93-4f68-4b2e-9cfc-9513c97edabf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32288d7e54cc49ea9260c4fbfa2e25a4'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:01.521875 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-8017dc93-4f68-4b2e-9cfc-9513c97edabf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32288d7e54cc49ea9260c4fbfa2e25a4'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:01.522792 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-8017dc93-4f68-4b2e-9cfc-9513c97edabf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32288d7e54cc49ea9260c4fbfa2e25a4'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:01.547135 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-8017dc93-4f68-4b2e-9cfc-9513c97edabf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32288d7e54cc49ea9260c4fbfa2e25a4'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:01.567223 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1350/2659] 10.222.0.66 () {58 vars in 1223 bytes} [Mon Jul 18 02:43:01 2022] DELETE /identity/v3/users/32288d7e54cc49ea9260c4fbfa2e25a4 => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:43:01.571632 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ce54be-8fba-47c6-bf9c-575699d7cf89 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:01.572846 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ce54be-8fba-47c6-bf9c-575699d7cf89 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:01.573278 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ce54be-8fba-47c6-bf9c-575699d7cf89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:01.612181 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-33ce54be-8fba-47c6-bf9c-575699d7cf89 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:43:01.614508 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-33ce54be-8fba-47c6-bf9c-575699d7cf89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:43:01.681278 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1310/2660] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:43:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:43:01.691855 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8017dc93-4f68-4b2e-9cfc-9513c97edabf admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:43:01.694211 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-8017dc93-4f68-4b2e-9cfc-9513c97edabf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:43:01.759315 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8017dc93-4f68-4b2e-9cfc-9513c97edabf admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:43:01.764764 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe2824e-1007-40e3-9ff9-69077c5a3b2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eaf19a259c2741cea375a38763e33cdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:43:01.765429 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe2824e-1007-40e3-9ff9-69077c5a3b2a service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:01.765714 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe2824e-1007-40e3-9ff9-69077c5a3b2a service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:01.765990 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe2824e-1007-40e3-9ff9-69077c5a3b2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:01.776340 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe2824e-1007-40e3-9ff9-69077c5a3b2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:43:01.777910 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe2824e-1007-40e3-9ff9-69077c5a3b2a service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:43:01.790461 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1351/2661] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:43:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:43:01.863601 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:43:02.336615 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ce54be-8fba-47c6-bf9c-575699d7cf89 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:43:02.338110 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-33ce54be-8fba-47c6-bf9c-575699d7cf89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:43:02.351174 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ce54be-8fba-47c6-bf9c-575699d7cf89 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:43:02.358930 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d741ee5-a93d-4308-b6e1-203828a20140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:43:02.360019 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d741ee5-a93d-4308-b6e1-203828a20140 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:02.360518 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d741ee5-a93d-4308-b6e1-203828a20140 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:02.361097 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d741ee5-a93d-4308-b6e1-203828a20140 admin admin] PATH_INFO: `/v3/projects/f5484a21aa1b4b378fa3e6e5f4fec572` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:02.363566 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d741ee5-a93d-4308-b6e1-203828a20140 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5484a21aa1b4b378fa3e6e5f4fec572)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:43:02.366263 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d741ee5-a93d-4308-b6e1-203828a20140 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:43:02.442840 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:43:02.526107 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-4d741ee5-a93d-4308-b6e1-203828a20140 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5484a21aa1b4b378fa3e6e5f4fec572', 'request_id': 'req-4d741ee5-a93d-4308-b6e1-203828a20140'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:02.533644 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-4d741ee5-a93d-4308-b6e1-203828a20140 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5484a21aa1b4b378fa3e6e5f4fec572', 'request_id': 'req-4d741ee5-a93d-4308-b6e1-203828a20140'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:02.541231 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-4d741ee5-a93d-4308-b6e1-203828a20140 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5484a21aa1b4b378fa3e6e5f4fec572', 'request_id': 'req-4d741ee5-a93d-4308-b6e1-203828a20140'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:02.639499 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-4d741ee5-a93d-4308-b6e1-203828a20140 admin admin] The token cache is being invalidate because project f5484a21aa1b4b378fa3e6e5f4fec572 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 02:43:02.640134 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-4d741ee5-a93d-4308-b6e1-203828a20140 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:02.640450 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-4d741ee5-a93d-4308-b6e1-203828a20140 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:02.642124 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1311/2662] 10.222.0.66 () {58 vars in 1232 bytes} [Mon Jul 18 02:43:02 2022] DELETE /identity/v3/projects/f5484a21aa1b4b378fa3e6e5f4fec572 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:43:02.647096 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50689dd2-734d-47e2-99cc-6144422f7b0d None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:02.647464 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50689dd2-734d-47e2-99cc-6144422f7b0d None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:02.647809 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50689dd2-734d-47e2-99cc-6144422f7b0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:02.705660 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-50689dd2-734d-47e2-99cc-6144422f7b0d None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:43:02.707444 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-50689dd2-734d-47e2-99cc-6144422f7b0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:43:02.777508 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1352/2663] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:43:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:43:02.785713 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace3a1e4-eb38-44c6-bf58-4edc10628bfa None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:02.786188 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace3a1e4-eb38-44c6-bf58-4edc10628bfa None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:02.786688 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace3a1e4-eb38-44c6-bf58-4edc10628bfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:02.806648 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-ace3a1e4-eb38-44c6-bf58-4edc10628bfa None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:43:02.808166 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-ace3a1e4-eb38-44c6-bf58-4edc10628bfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:43:02.820164 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1312/2664] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:43:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 02:43:02.827022 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50689dd2-734d-47e2-99cc-6144422f7b0d None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:43:02.838869 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50689dd2-734d-47e2-99cc-6144422f7b0d None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:43:02.845114 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eb8fea-b0fd-4a6f-b3a5-b0edaddcde33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:43:02.845882 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eb8fea-b0fd-4a6f-b3a5-b0edaddcde33 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:02.846289 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eb8fea-b0fd-4a6f-b3a5-b0edaddcde33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:02.846639 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eb8fea-b0fd-4a6f-b3a5-b0edaddcde33 admin admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:02.847458 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3eb8fea-b0fd-4a6f-b3a5-b0edaddcde33 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:43:02.848022 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3eb8fea-b0fd-4a6f-b3a5-b0edaddcde33 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:43:02.849570 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3eb8fea-b0fd-4a6f-b3a5-b0edaddcde33 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:43:02.864699 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1353/2665] 10.222.0.66 () {58 vars in 1152 bytes} [Mon Jul 18 02:43:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:43:02.879161 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace3a1e4-eb38-44c6-bf58-4edc10628bfa None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:43:02.900065 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace3a1e4-eb38-44c6-bf58-4edc10628bfa None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:43:02.905032 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0b6717-6fac-4363-bceb-b3c180769bca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:43:02.905761 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0b6717-6fac-4363-bceb-b3c180769bca admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:02.906009 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0b6717-6fac-4363-bceb-b3c180769bca admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:02.906327 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0b6717-6fac-4363-bceb-b3c180769bca admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:02.907035 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0b6717-6fac-4363-bceb-b3c180769bca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:43:02.909728 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0b6717-6fac-4363-bceb-b3c180769bca admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:43:03.014332 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1313/2666] 10.222.0.66 () {60 vars in 1152 bytes} [Mon Jul 18 02:43:02 2022] POST /identity/v3/projects => generated 373 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:43:03.018182 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eb8fea-b0fd-4a6f-b3a5-b0edaddcde33 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:43:03.029419 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eb8fea-b0fd-4a6f-b3a5-b0edaddcde33 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:43:03.097484 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82224741-23f0-4871-8953-30cecf8c11ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:43:03.098272 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82224741-23f0-4871-8953-30cecf8c11ba admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:03.101349 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82224741-23f0-4871-8953-30cecf8c11ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:03.101349 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82224741-23f0-4871-8953-30cecf8c11ba admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:03.101349 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82224741-23f0-4871-8953-30cecf8c11ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:43:03.101738 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82224741-23f0-4871-8953-30cecf8c11ba admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:43:03.113673 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9351a326-5fee-43f9-96fc-098acd4fed13 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:03.113893 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9351a326-5fee-43f9-96fc-098acd4fed13 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:03.114215 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9351a326-5fee-43f9-96fc-098acd4fed13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:03.142286 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-9351a326-5fee-43f9-96fc-098acd4fed13 None None] MFA Rules not processed for user `cb4702f0216b47b8ba83872eeee40173`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:43:03.143596 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-9351a326-5fee-43f9-96fc-098acd4fed13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:43:03.153159 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:43:03.217502 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1314/2667] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:43:03 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 02:43:03.233224 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9351a326-5fee-43f9-96fc-098acd4fed13 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:43:03.234745 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-9351a326-5fee-43f9-96fc-098acd4fed13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:43:03.242378 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1354/2668] 10.222.0.66 () {60 vars in 1143 bytes} [Mon Jul 18 02:43:03 2022] POST /identity/v3/users => generated 287 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:43:03.247649 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82224741-23f0-4871-8953-30cecf8c11ba admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:43:03.257751 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82224741-23f0-4871-8953-30cecf8c11ba admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:43:03.265173 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76654b1e-7242-464f-b0a0-4559a15cd227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:43:03.265944 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76654b1e-7242-464f-b0a0-4559a15cd227 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:03.266435 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76654b1e-7242-464f-b0a0-4559a15cd227 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:03.266779 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76654b1e-7242-464f-b0a0-4559a15cd227 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:03.272482 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76654b1e-7242-464f-b0a0-4559a15cd227 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:43:03.272482 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76654b1e-7242-464f-b0a0-4559a15cd227 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:43:03.282839 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1355/2669] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:43:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 02:43:03.283782 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76654b1e-7242-464f-b0a0-4559a15cd227 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:43:03.293428 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76654b1e-7242-464f-b0a0-4559a15cd227 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:43:03.298800 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1e24ea-50b5-480f-913b-5050d8b0e5c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:43:03.299453 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1e24ea-50b5-480f-913b-5050d8b0e5c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:03.299739 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1e24ea-50b5-480f-913b-5050d8b0e5c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:03.300018 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1e24ea-50b5-480f-913b-5050d8b0e5c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:03.300733 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1e24ea-50b5-480f-913b-5050d8b0e5c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:43:03.302372 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1e24ea-50b5-480f-913b-5050d8b0e5c3 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:43:03.313304 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1356/2670] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:43:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 02:43:03.317338 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1e24ea-50b5-480f-913b-5050d8b0e5c3 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:43:03.327268 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1e24ea-50b5-480f-913b-5050d8b0e5c3 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:43:03.331903 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a99fe52-3a04-4120-bb83-5c784912e201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:43:03.332774 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a99fe52-3a04-4120-bb83-5c784912e201 admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:03.333059 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a99fe52-3a04-4120-bb83-5c784912e201 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:03.333336 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a99fe52-3a04-4120-bb83-5c784912e201 admin admin] PATH_INFO: `/v3/projects/0a03054a3a804bb78fc6c3656281f3cd/users/f17edc7509384346b459bc4e3f92a5f0/roles/65feabf3b27944a8b01a86c8ac608457` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:03.338639 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9351a326-5fee-43f9-96fc-098acd4fed13 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:43:03.345198 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1117f56-f5f1-4ca7-8f16-fefa64cf1ca2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80b138831bc54c7b986d50092d62d4af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:43:03.346350 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1117f56-f5f1-4ca7-8f16-fefa64cf1ca2 service nova] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:03.346671 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1117f56-f5f1-4ca7-8f16-fefa64cf1ca2 service nova] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:03.346957 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1117f56-f5f1-4ca7-8f16-fefa64cf1ca2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:03.348837 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-b1117f56-f5f1-4ca7-8f16-fefa64cf1ca2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:43:03.357349 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a99fe52-3a04-4120-bb83-5c784912e201 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a03054a3a804bb78fc6c3656281f3cd, user_id=f17edc7509384346b459bc4e3f92a5f0, role_id=65feabf3b27944a8b01a86c8ac608457)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:43:03.357349 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a99fe52-3a04-4120-bb83-5c784912e201 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:43:03.408555 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1117f56-f5f1-4ca7-8f16-fefa64cf1ca2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:43:03.410203 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1117f56-f5f1-4ca7-8f16-fefa64cf1ca2 service nova] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:43:03.417423 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1357/2671] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:43:03 2022] PUT /identity/v3/projects/0a03054a3a804bb78fc6c3656281f3cd/users/f17edc7509384346b459bc4e3f92a5f0/roles/65feabf3b27944a8b01a86c8ac608457 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 02:43:03.427324 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1315/2672] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:43:03 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 02:43:03.428842 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcc486d-bd22-4889-b7d0-1ca74b5287c8 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:03.429224 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcc486d-bd22-4889-b7d0-1ca74b5287c8 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:03.429568 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcc486d-bd22-4889-b7d0-1ca74b5287c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:03.457387 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-9fcc486d-bd22-4889-b7d0-1ca74b5287c8 None None] MFA Rules not processed for user `f17edc7509384346b459bc4e3f92a5f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:43:03.458636 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-9fcc486d-bd22-4889-b7d0-1ca74b5287c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:43:03.526510 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1358/2673] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:43:03 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:43:03.539502 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1117f56-f5f1-4ca7-8f16-fefa64cf1ca2 service nova] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:43:03.541018 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-b1117f56-f5f1-4ca7-8f16-fefa64cf1ca2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:43:03.598917 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1117f56-f5f1-4ca7-8f16-fefa64cf1ca2 service nova] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:43:03.647516 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615f592f-a267-4f75-b3a6-8ad13f11e9d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd49388facdf446885e55b45ef91f01e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:43:03.648171 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615f592f-a267-4f75-b3a6-8ad13f11e9d6 service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:03.648455 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615f592f-a267-4f75-b3a6-8ad13f11e9d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:03.648732 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615f592f-a267-4f75-b3a6-8ad13f11e9d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:03.658272 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615f592f-a267-4f75-b3a6-8ad13f11e9d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:43:03.659755 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615f592f-a267-4f75-b3a6-8ad13f11e9d6 service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:43:03.668906 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1316/2674] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:43:03 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:43:03.694707 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcc486d-bd22-4889-b7d0-1ca74b5287c8 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:43:03.760804 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcc486d-bd22-4889-b7d0-1ca74b5287c8 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:43:03.768997 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:43:03.770963 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269a2722-b6ab-4abb-b980-0133c55ec063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:43:03.772034 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269a2722-b6ab-4abb-b980-0133c55ec063 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:03.782659 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269a2722-b6ab-4abb-b980-0133c55ec063 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:03.782659 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269a2722-b6ab-4abb-b980-0133c55ec063 admin admin] PATH_INFO: `/v3/users/f17edc7509384346b459bc4e3f92a5f0` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:03.782659 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269a2722-b6ab-4abb-b980-0133c55ec063 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f17edc7509384346b459bc4e3f92a5f0)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:43:03.782659 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269a2722-b6ab-4abb-b980-0133c55ec063 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:43:04.025433 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-269a2722-b6ab-4abb-b980-0133c55ec063 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f17edc7509384346b459bc4e3f92a5f0'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:04.031631 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:43:04.109200 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-269a2722-b6ab-4abb-b980-0133c55ec063 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f17edc7509384346b459bc4e3f92a5f0'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:04.116681 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-269a2722-b6ab-4abb-b980-0133c55ec063 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f17edc7509384346b459bc4e3f92a5f0'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:04.117344 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-269a2722-b6ab-4abb-b980-0133c55ec063 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f17edc7509384346b459bc4e3f92a5f0'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:04.130908 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-269a2722-b6ab-4abb-b980-0133c55ec063 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f17edc7509384346b459bc4e3f92a5f0'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:04.138316 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1359/2675] 10.222.0.66 () {58 vars in 1223 bytes} [Mon Jul 18 02:43:03 2022] DELETE /identity/v3/users/f17edc7509384346b459bc4e3f92a5f0 => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:43:04.142669 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bf89b7-60e9-46f7-b0bd-9e61515e3c1c None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:04.142959 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bf89b7-60e9-46f7-b0bd-9e61515e3c1c None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:04.143235 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bf89b7-60e9-46f7-b0bd-9e61515e3c1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:04.163784 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-d6bf89b7-60e9-46f7-b0bd-9e61515e3c1c None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:43:04.165139 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-d6bf89b7-60e9-46f7-b0bd-9e61515e3c1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:43:04.233054 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1317/2676] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:43:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:43:04.240256 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269a2722-b6ab-4abb-b980-0133c55ec063 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:43:04.242558 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-269a2722-b6ab-4abb-b980-0133c55ec063 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:43:04.300467 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269a2722-b6ab-4abb-b980-0133c55ec063 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:43:04.304110 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2568fa8-43c3-4f5b-baa2-eeb2e2a06ff2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eaf19a259c2741cea375a38763e33cdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:43:04.304768 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2568fa8-43c3-4f5b-baa2-eeb2e2a06ff2 service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:04.305056 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2568fa8-43c3-4f5b-baa2-eeb2e2a06ff2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:04.305336 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2568fa8-43c3-4f5b-baa2-eeb2e2a06ff2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:04.315508 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2568fa8-43c3-4f5b-baa2-eeb2e2a06ff2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:43:04.316963 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2568fa8-43c3-4f5b-baa2-eeb2e2a06ff2 service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:43:04.327777 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1360/2677] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:43:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:43:04.639501 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:43:04.940521 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bf89b7-60e9-46f7-b0bd-9e61515e3c1c None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:43:04.941970 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-d6bf89b7-60e9-46f7-b0bd-9e61515e3c1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:43:04.955997 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bf89b7-60e9-46f7-b0bd-9e61515e3c1c None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:43:04.961546 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0778693-c16e-4673-83a3-b89050a8a6cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:43:04.962644 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0778693-c16e-4673-83a3-b89050a8a6cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:04.962762 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0778693-c16e-4673-83a3-b89050a8a6cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:04.963281 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0778693-c16e-4673-83a3-b89050a8a6cb admin admin] PATH_INFO: `/v3/projects/0a03054a3a804bb78fc6c3656281f3cd` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:04.964486 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0778693-c16e-4673-83a3-b89050a8a6cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a03054a3a804bb78fc6c3656281f3cd)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:43:04.967225 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0778693-c16e-4673-83a3-b89050a8a6cb admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:43:05.037888 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2568fa8-43c3-4f5b-baa2-eeb2e2a06ff2 service neutron] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:43:05.040049 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-d2568fa8-43c3-4f5b-baa2-eeb2e2a06ff2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:43:05.093861 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-c0778693-c16e-4673-83a3-b89050a8a6cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a03054a3a804bb78fc6c3656281f3cd', 'request_id': 'req-c0778693-c16e-4673-83a3-b89050a8a6cb'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:05.100558 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-c0778693-c16e-4673-83a3-b89050a8a6cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a03054a3a804bb78fc6c3656281f3cd', 'request_id': 'req-c0778693-c16e-4673-83a3-b89050a8a6cb'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:05.106953 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-c0778693-c16e-4673-83a3-b89050a8a6cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a03054a3a804bb78fc6c3656281f3cd', 'request_id': 'req-c0778693-c16e-4673-83a3-b89050a8a6cb'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:05.108409 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2568fa8-43c3-4f5b-baa2-eeb2e2a06ff2 service neutron] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:43:05.161308 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508b1830-5aa1-42f3-9373-74824af4792f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd49388facdf446885e55b45ef91f01e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:43:05.162104 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508b1830-5aa1-42f3-9373-74824af4792f service cinder] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:05.162411 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508b1830-5aa1-42f3-9373-74824af4792f service cinder] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:05.162755 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508b1830-5aa1-42f3-9373-74824af4792f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:05.164621 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-508b1830-5aa1-42f3-9373-74824af4792f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:43:05.179876 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-c0778693-c16e-4673-83a3-b89050a8a6cb admin admin] The token cache is being invalidate because project 0a03054a3a804bb78fc6c3656281f3cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 02:43:05.180552 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-c0778693-c16e-4673-83a3-b89050a8a6cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:05.181769 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-c0778693-c16e-4673-83a3-b89050a8a6cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:05.184457 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1318/2678] 10.222.0.66 () {58 vars in 1232 bytes} [Mon Jul 18 02:43:04 2022] DELETE /identity/v3/projects/0a03054a3a804bb78fc6c3656281f3cd => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:43:05.189887 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f509f510-d787-41e0-aa29-b0da48022bd6 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:05.190749 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f509f510-d787-41e0-aa29-b0da48022bd6 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:05.191035 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f509f510-d787-41e0-aa29-b0da48022bd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:05.211254 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-f509f510-d787-41e0-aa29-b0da48022bd6 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:43:05.212477 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-f509f510-d787-41e0-aa29-b0da48022bd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:43:05.247963 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508b1830-5aa1-42f3-9373-74824af4792f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:43:05.250124 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508b1830-5aa1-42f3-9373-74824af4792f service cinder] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:43:05.263556 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1361/2679] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:43:05 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 02:43:05.329752 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1319/2680] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:43:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:43:05.338685 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd8f5ee-5626-4b60-9176-4aff72114209 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:05.340142 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd8f5ee-5626-4b60-9176-4aff72114209 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:05.340442 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd8f5ee-5626-4b60-9176-4aff72114209 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:05.362960 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-9dd8f5ee-5626-4b60-9176-4aff72114209 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:43:05.364365 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-9dd8f5ee-5626-4b60-9176-4aff72114209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:43:05.375496 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1362/2681] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:43:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 02:43:05.380197 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f509f510-d787-41e0-aa29-b0da48022bd6 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:43:05.409845 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f509f510-d787-41e0-aa29-b0da48022bd6 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:43:05.415381 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71ed510-b2f4-420e-b866-695f1da449d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:43:05.416304 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71ed510-b2f4-420e-b866-695f1da449d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:05.416659 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71ed510-b2f4-420e-b866-695f1da449d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:05.417041 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71ed510-b2f4-420e-b866-695f1da449d0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:05.417966 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71ed510-b2f4-420e-b866-695f1da449d0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:43:05.418375 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71ed510-b2f4-420e-b866-695f1da449d0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:43:05.421156 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71ed510-b2f4-420e-b866-695f1da449d0 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:43:05.437539 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1320/2682] 10.222.0.66 () {58 vars in 1152 bytes} [Mon Jul 18 02:43:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:43:05.442021 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd8f5ee-5626-4b60-9176-4aff72114209 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:43:05.455988 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd8f5ee-5626-4b60-9176-4aff72114209 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:43:05.464020 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420b6bfd-7123-473d-bb73-9c0635bf0cc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:43:05.464680 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420b6bfd-7123-473d-bb73-9c0635bf0cc8 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:05.465324 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420b6bfd-7123-473d-bb73-9c0635bf0cc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:05.465794 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420b6bfd-7123-473d-bb73-9c0635bf0cc8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:05.466693 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420b6bfd-7123-473d-bb73-9c0635bf0cc8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:43:05.470670 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420b6bfd-7123-473d-bb73-9c0635bf0cc8 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:43:05.492033 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:43:05.574114 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1363/2683] 10.222.0.66 () {60 vars in 1152 bytes} [Mon Jul 18 02:43:05 2022] POST /identity/v3/projects => generated 373 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:43:05.577563 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71ed510-b2f4-420e-b866-695f1da449d0 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:43:05.587980 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71ed510-b2f4-420e-b866-695f1da449d0 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:43:05.650240 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41450b1c-daf6-4985-bbb2-06f54e4c4f52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:43:05.650916 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41450b1c-daf6-4985-bbb2-06f54e4c4f52 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:05.651204 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41450b1c-daf6-4985-bbb2-06f54e4c4f52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:05.651483 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41450b1c-daf6-4985-bbb2-06f54e4c4f52 admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:05.652188 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41450b1c-daf6-4985-bbb2-06f54e4c4f52 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:43:05.654335 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41450b1c-daf6-4985-bbb2-06f54e4c4f52 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:43:05.728184 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1321/2684] 10.222.0.66 () {60 vars in 1143 bytes} [Mon Jul 18 02:43:05 2022] POST /identity/v3/users => generated 287 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 02:43:05.742022 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420b6bfd-7123-473d-bb73-9c0635bf0cc8 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:43:05.742893 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420b6bfd-7123-473d-bb73-9c0635bf0cc8 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:43:05.748694 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ccea19-982d-4c12-8c0b-483ccb2f2282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:43:05.749619 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ccea19-982d-4c12-8c0b-483ccb2f2282 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:05.750001 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ccea19-982d-4c12-8c0b-483ccb2f2282 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:05.750379 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ccea19-982d-4c12-8c0b-483ccb2f2282 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:05.751545 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ccea19-982d-4c12-8c0b-483ccb2f2282 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:43:05.753312 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ccea19-982d-4c12-8c0b-483ccb2f2282 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:43:05.762387 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:43:05.775916 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1364/2685] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:43:05 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 02:43:05.779889 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41450b1c-daf6-4985-bbb2-06f54e4c4f52 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:43:05.805257 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41450b1c-daf6-4985-bbb2-06f54e4c4f52 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:43:05.816674 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c052fe6c-6232-4b97-92af-bc0a83a985e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:43:05.816674 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c052fe6c-6232-4b97-92af-bc0a83a985e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:05.816674 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c052fe6c-6232-4b97-92af-bc0a83a985e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:05.816674 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c052fe6c-6232-4b97-92af-bc0a83a985e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:05.817841 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c052fe6c-6232-4b97-92af-bc0a83a985e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:43:05.817841 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c052fe6c-6232-4b97-92af-bc0a83a985e2 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:43:05.829819 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1322/2686] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:43:05 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 02:43:05.834115 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ccea19-982d-4c12-8c0b-483ccb2f2282 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:43:05.847402 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ccea19-982d-4c12-8c0b-483ccb2f2282 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:43:05.851581 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4fa4c7-cfb1-41cb-a902-e2d1509c86f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:43:05.852508 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4fa4c7-cfb1-41cb-a902-e2d1509c86f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:05.852790 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4fa4c7-cfb1-41cb-a902-e2d1509c86f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:05.853072 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4fa4c7-cfb1-41cb-a902-e2d1509c86f8 admin admin] PATH_INFO: `/v3/projects/84eacffbd4ec4b2e9d80c38ab47f02d2/users/5b57b0b4a0044a70856aa886c7188521/roles/65feabf3b27944a8b01a86c8ac608457` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:05.873136 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4fa4c7-cfb1-41cb-a902-e2d1509c86f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=84eacffbd4ec4b2e9d80c38ab47f02d2, user_id=5b57b0b4a0044a70856aa886c7188521, role_id=65feabf3b27944a8b01a86c8ac608457)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:43:05.876448 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4fa4c7-cfb1-41cb-a902-e2d1509c86f8 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:43:05.957765 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1365/2687] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:43:05 2022] PUT /identity/v3/projects/84eacffbd4ec4b2e9d80c38ab47f02d2/users/5b57b0b4a0044a70856aa886c7188521/roles/65feabf3b27944a8b01a86c8ac608457 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:43:05.964414 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d39c28-ad7f-42d7-8cc8-7171e72f0655 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:05.964937 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d39c28-ad7f-42d7-8cc8-7171e72f0655 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:05.964937 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d39c28-ad7f-42d7-8cc8-7171e72f0655 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:05.991889 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-99d39c28-ad7f-42d7-8cc8-7171e72f0655 None None] MFA Rules not processed for user `5b57b0b4a0044a70856aa886c7188521`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:43:05.993107 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-99d39c28-ad7f-42d7-8cc8-7171e72f0655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:43:06.071994 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1323/2688] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:43:05 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 02:43:06.086771 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4fa4c7-cfb1-41cb-a902-e2d1509c86f8 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:43:06.088925 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-8d4fa4c7-cfb1-41cb-a902-e2d1509c86f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:43:06.171120 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4fa4c7-cfb1-41cb-a902-e2d1509c86f8 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:43:06.188573 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc11773d-dd62-49e4-b5eb-939566d82cbc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd49388facdf446885e55b45ef91f01e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:43:06.195958 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc11773d-dd62-49e4-b5eb-939566d82cbc service cinder] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:06.195958 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc11773d-dd62-49e4-b5eb-939566d82cbc service cinder] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:06.195958 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc11773d-dd62-49e4-b5eb-939566d82cbc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:06.204308 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc11773d-dd62-49e4-b5eb-939566d82cbc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:43:06.207039 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc11773d-dd62-49e4-b5eb-939566d82cbc service cinder] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:43:06.223222 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1366/2689] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:43:06 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:43:06.432850 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac263f82-92e5-4ee4-9b33-2c022257742f None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:06.433150 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac263f82-92e5-4ee4-9b33-2c022257742f None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:06.433605 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac263f82-92e5-4ee4-9b33-2c022257742f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:06.457507 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-ac263f82-92e5-4ee4-9b33-2c022257742f None None] MFA Rules not processed for user `681d0f989d7d459e83a32066fd5ed0a2`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:43:06.459162 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-ac263f82-92e5-4ee4-9b33-2c022257742f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:43:06.538216 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1324/2690] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:43:06 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 02:43:06.938151 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:43:07.503811 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:43:08.063847 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:43:08.538207 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:43:08.908917 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:43:08.926049 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:43:09.011794 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:43:09.034670 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:43:09.166733 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:43:09.193704 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:43:09.235332 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:43:10.574361 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc11773d-dd62-49e4-b5eb-939566d82cbc service cinder] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:43:10.587914 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc11773d-dd62-49e4-b5eb-939566d82cbc service cinder] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:43:10.594457 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680e10dd-10e8-47ef-967b-ca48a61030bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd49388facdf446885e55b45ef91f01e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:43:10.594457 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680e10dd-10e8-47ef-967b-ca48a61030bb service cinder] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:10.594457 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680e10dd-10e8-47ef-967b-ca48a61030bb service cinder] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:10.594457 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680e10dd-10e8-47ef-967b-ca48a61030bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:10.594842 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-680e10dd-10e8-47ef-967b-ca48a61030bb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:43:10.668668 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680e10dd-10e8-47ef-967b-ca48a61030bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:43:10.670511 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680e10dd-10e8-47ef-967b-ca48a61030bb service cinder] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:43:10.680699 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1367/2691] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:43:10 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:43:19.725595 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=55 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:43:20.675875 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=88,keystone:INSERT=2 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:43:23.379027 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac263f82-92e5-4ee4-9b33-2c022257742f None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:43:23.380515 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-ac263f82-92e5-4ee4-9b33-2c022257742f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:43:23.447232 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac263f82-92e5-4ee4-9b33-2c022257742f None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:43:23.452798 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba1085a-80b9-43a6-99fc-3b55a6a150b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:43:23.453682 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba1085a-80b9-43a6-99fc-3b55a6a150b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:23.453969 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba1085a-80b9-43a6-99fc-3b55a6a150b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:23.454272 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba1085a-80b9-43a6-99fc-3b55a6a150b7 admin admin] PATH_INFO: `/v3/users/a164da80f13446f0aa26527aeeb638ab` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:23.455481 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba1085a-80b9-43a6-99fc-3b55a6a150b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a164da80f13446f0aa26527aeeb638ab)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:43:23.457431 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba1085a-80b9-43a6-99fc-3b55a6a150b7 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:43:23.834384 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-eba1085a-80b9-43a6-99fc-3b55a6a150b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a164da80f13446f0aa26527aeeb638ab'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:24.005993 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-eba1085a-80b9-43a6-99fc-3b55a6a150b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a164da80f13446f0aa26527aeeb638ab'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:24.006698 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-eba1085a-80b9-43a6-99fc-3b55a6a150b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a164da80f13446f0aa26527aeeb638ab'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:24.020043 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-eba1085a-80b9-43a6-99fc-3b55a6a150b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a164da80f13446f0aa26527aeeb638ab'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:24.020681 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-eba1085a-80b9-43a6-99fc-3b55a6a150b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a164da80f13446f0aa26527aeeb638ab'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:24.034336 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1325/2692] 10.222.0.66 () {58 vars in 1223 bytes} [Mon Jul 18 02:43:23 2022] DELETE /identity/v3/users/a164da80f13446f0aa26527aeeb638ab => generated 0 bytes in 656 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:43:24.038851 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cdec4a-152c-40d3-892c-39fb090dabff None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:24.039389 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cdec4a-152c-40d3-892c-39fb090dabff None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:24.045935 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cdec4a-152c-40d3-892c-39fb090dabff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:24.058292 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-21cdec4a-152c-40d3-892c-39fb090dabff None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:43:24.059869 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-21cdec4a-152c-40d3-892c-39fb090dabff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:43:24.126425 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1368/2693] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:43:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:43:24.141794 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba1085a-80b9-43a6-99fc-3b55a6a150b7 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:43:24.143300 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-eba1085a-80b9-43a6-99fc-3b55a6a150b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:43:24.237255 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba1085a-80b9-43a6-99fc-3b55a6a150b7 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:43:24.241106 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db4b28a-c818-485e-a44a-b52678b8797d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eaf19a259c2741cea375a38763e33cdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:43:24.241777 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db4b28a-c818-485e-a44a-b52678b8797d service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:24.242064 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db4b28a-c818-485e-a44a-b52678b8797d service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:24.242378 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db4b28a-c818-485e-a44a-b52678b8797d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:24.254153 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db4b28a-c818-485e-a44a-b52678b8797d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:43:24.255604 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db4b28a-c818-485e-a44a-b52678b8797d service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:43:24.266822 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1326/2694] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:43:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:43:24.762381 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cdec4a-152c-40d3-892c-39fb090dabff None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:43:24.764159 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-21cdec4a-152c-40d3-892c-39fb090dabff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:43:24.782680 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cdec4a-152c-40d3-892c-39fb090dabff None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:43:24.787849 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ac5c10-a1de-4723-afca-27fb3c6385e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:43:24.788756 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ac5c10-a1de-4723-afca-27fb3c6385e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:24.789205 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ac5c10-a1de-4723-afca-27fb3c6385e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:24.789303 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ac5c10-a1de-4723-afca-27fb3c6385e4 admin admin] PATH_INFO: `/v3/projects/5ef7150f06aa4cbdab1ca9407581a50d` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:24.790610 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ac5c10-a1de-4723-afca-27fb3c6385e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ef7150f06aa4cbdab1ca9407581a50d)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:43:24.792589 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ac5c10-a1de-4723-afca-27fb3c6385e4 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:43:24.969425 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-e5ac5c10-a1de-4723-afca-27fb3c6385e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ef7150f06aa4cbdab1ca9407581a50d', 'request_id': 'req-e5ac5c10-a1de-4723-afca-27fb3c6385e4'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:24.975794 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-e5ac5c10-a1de-4723-afca-27fb3c6385e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ef7150f06aa4cbdab1ca9407581a50d', 'request_id': 'req-e5ac5c10-a1de-4723-afca-27fb3c6385e4'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:25.080398 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-e5ac5c10-a1de-4723-afca-27fb3c6385e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ef7150f06aa4cbdab1ca9407581a50d', 'request_id': 'req-e5ac5c10-a1de-4723-afca-27fb3c6385e4'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:25.092777 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-e5ac5c10-a1de-4723-afca-27fb3c6385e4 admin admin] The token cache is being invalidate because project 5ef7150f06aa4cbdab1ca9407581a50d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 02:43:25.093058 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-e5ac5c10-a1de-4723-afca-27fb3c6385e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:25.093764 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-e5ac5c10-a1de-4723-afca-27fb3c6385e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:25.095331 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1369/2695] 10.222.0.66 () {58 vars in 1232 bytes} [Mon Jul 18 02:43:24 2022] DELETE /identity/v3/projects/5ef7150f06aa4cbdab1ca9407581a50d => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:43:25.099432 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db4b28a-c818-485e-a44a-b52678b8797d service neutron] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:43:25.104416 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:43:25.110413 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db4b28a-c818-485e-a44a-b52678b8797d service neutron] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:43:25.165687 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bdffe5-889c-496a-b6da-0464211e3980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:43:25.166597 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bdffe5-889c-496a-b6da-0464211e3980 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:25.166887 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bdffe5-889c-496a-b6da-0464211e3980 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:25.167168 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bdffe5-889c-496a-b6da-0464211e3980 admin admin] PATH_INFO: `/v3/users/e4f5da7039104233b2d7d4c107b81140` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:25.168352 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bdffe5-889c-496a-b6da-0464211e3980 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4f5da7039104233b2d7d4c107b81140)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:43:25.172104 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bdffe5-889c-496a-b6da-0464211e3980 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:43:25.355419 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ac5c10-a1de-4723-afca-27fb3c6385e4 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:43:25.356899 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-e5ac5c10-a1de-4723-afca-27fb3c6385e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:43:25.370208 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ac5c10-a1de-4723-afca-27fb3c6385e4 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:43:25.374854 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012007c6-67b4-4f7b-852e-01470b2edeeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:43:25.375720 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012007c6-67b4-4f7b-852e-01470b2edeeb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:25.376043 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012007c6-67b4-4f7b-852e-01470b2edeeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:25.376504 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012007c6-67b4-4f7b-852e-01470b2edeeb admin admin] PATH_INFO: `/v3/users/5b57b0b4a0044a70856aa886c7188521` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:25.377755 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012007c6-67b4-4f7b-852e-01470b2edeeb admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b57b0b4a0044a70856aa886c7188521)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:43:25.379851 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012007c6-67b4-4f7b-852e-01470b2edeeb admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:43:25.510675 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-a5bdffe5-889c-496a-b6da-0464211e3980 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4f5da7039104233b2d7d4c107b81140'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:25.550770 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-012007c6-67b4-4f7b-852e-01470b2edeeb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b57b0b4a0044a70856aa886c7188521'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:25.623365 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-a5bdffe5-889c-496a-b6da-0464211e3980 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4f5da7039104233b2d7d4c107b81140'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:25.624072 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-a5bdffe5-889c-496a-b6da-0464211e3980 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4f5da7039104233b2d7d4c107b81140'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:25.637550 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-a5bdffe5-889c-496a-b6da-0464211e3980 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4f5da7039104233b2d7d4c107b81140'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:25.638932 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-a5bdffe5-889c-496a-b6da-0464211e3980 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4f5da7039104233b2d7d4c107b81140'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:25.652050 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1327/2696] 10.222.0.66 () {58 vars in 1223 bytes} [Mon Jul 18 02:43:25 2022] DELETE /identity/v3/users/e4f5da7039104233b2d7d4c107b81140 => generated 0 bytes in 553 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:43:25.686451 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-012007c6-67b4-4f7b-852e-01470b2edeeb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b57b0b4a0044a70856aa886c7188521'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:25.694421 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-012007c6-67b4-4f7b-852e-01470b2edeeb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b57b0b4a0044a70856aa886c7188521'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:25.695119 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-012007c6-67b4-4f7b-852e-01470b2edeeb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b57b0b4a0044a70856aa886c7188521'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:25.707808 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:43:25.713447 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-012007c6-67b4-4f7b-852e-01470b2edeeb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b57b0b4a0044a70856aa886c7188521'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:25.715520 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1370/2697] 10.222.0.66 () {58 vars in 1223 bytes} [Mon Jul 18 02:43:25 2022] DELETE /identity/v3/users/5b57b0b4a0044a70856aa886c7188521 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 02:43:25.720293 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272a1071-4390-4ef7-8b22-e6a8a5b1c746 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:25.720586 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272a1071-4390-4ef7-8b22-e6a8a5b1c746 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:25.720890 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272a1071-4390-4ef7-8b22-e6a8a5b1c746 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:25.742184 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-272a1071-4390-4ef7-8b22-e6a8a5b1c746 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:43:25.743420 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-272a1071-4390-4ef7-8b22-e6a8a5b1c746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:43:25.804316 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1328/2698] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:43:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:43:25.814940 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012007c6-67b4-4f7b-852e-01470b2edeeb admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:43:25.817556 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-012007c6-67b4-4f7b-852e-01470b2edeeb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:43:25.878498 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012007c6-67b4-4f7b-852e-01470b2edeeb admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:43:25.882479 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf47b05-f5d3-4440-9a0c-0db945aa434a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eaf19a259c2741cea375a38763e33cdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:43:25.883189 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf47b05-f5d3-4440-9a0c-0db945aa434a service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:25.883480 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf47b05-f5d3-4440-9a0c-0db945aa434a service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:25.883778 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf47b05-f5d3-4440-9a0c-0db945aa434a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:25.894008 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf47b05-f5d3-4440-9a0c-0db945aa434a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:43:25.895479 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf47b05-f5d3-4440-9a0c-0db945aa434a service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:43:25.905414 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1371/2699] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:43:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:43:26.070933 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272a1071-4390-4ef7-8b22-e6a8a5b1c746 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:43:26.072445 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-272a1071-4390-4ef7-8b22-e6a8a5b1c746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:43:26.085620 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272a1071-4390-4ef7-8b22-e6a8a5b1c746 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:43:26.090313 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd8f0e8-6de4-4a97-aa53-ed9ce7eeb489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:43:26.092025 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd8f0e8-6de4-4a97-aa53-ed9ce7eeb489 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:26.092373 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd8f0e8-6de4-4a97-aa53-ed9ce7eeb489 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:26.092718 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd8f0e8-6de4-4a97-aa53-ed9ce7eeb489 admin admin] PATH_INFO: `/v3/projects/8eab6f0a2a46471f9aa267084e9e40fe` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:26.094137 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd8f0e8-6de4-4a97-aa53-ed9ce7eeb489 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8eab6f0a2a46471f9aa267084e9e40fe)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:43:26.096384 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd8f0e8-6de4-4a97-aa53-ed9ce7eeb489 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:43:26.115407 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:43:26.231473 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-9dd8f0e8-6de4-4a97-aa53-ed9ce7eeb489 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8eab6f0a2a46471f9aa267084e9e40fe', 'request_id': 'req-9dd8f0e8-6de4-4a97-aa53-ed9ce7eeb489'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:26.238604 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-9dd8f0e8-6de4-4a97-aa53-ed9ce7eeb489 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8eab6f0a2a46471f9aa267084e9e40fe', 'request_id': 'req-9dd8f0e8-6de4-4a97-aa53-ed9ce7eeb489'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:26.251867 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-9dd8f0e8-6de4-4a97-aa53-ed9ce7eeb489 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8eab6f0a2a46471f9aa267084e9e40fe', 'request_id': 'req-9dd8f0e8-6de4-4a97-aa53-ed9ce7eeb489'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:26.365696 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-9dd8f0e8-6de4-4a97-aa53-ed9ce7eeb489 admin admin] The token cache is being invalidate because project 8eab6f0a2a46471f9aa267084e9e40fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 02:43:26.365989 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-9dd8f0e8-6de4-4a97-aa53-ed9ce7eeb489 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:26.366656 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-9dd8f0e8-6de4-4a97-aa53-ed9ce7eeb489 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:26.368297 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1329/2700] 10.222.0.66 () {58 vars in 1232 bytes} [Mon Jul 18 02:43:26 2022] DELETE /identity/v3/projects/8eab6f0a2a46471f9aa267084e9e40fe => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:43:26.374895 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a5a0a1-b953-4be8-9d10-f0acf068318b None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:26.375419 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a5a0a1-b953-4be8-9d10-f0acf068318b None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:26.375524 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a5a0a1-b953-4be8-9d10-f0acf068318b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:26.394435 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-05a5a0a1-b953-4be8-9d10-f0acf068318b None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:43:26.395629 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-05a5a0a1-b953-4be8-9d10-f0acf068318b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:43:26.413968 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd8f0e8-6de4-4a97-aa53-ed9ce7eeb489 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:43:26.415872 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-9dd8f0e8-6de4-4a97-aa53-ed9ce7eeb489 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:43:26.462773 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1372/2701] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:43:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 02:43:26.471051 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf872d2-09bd-47b7-912c-05fe4ff1c8df None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:26.471337 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf872d2-09bd-47b7-912c-05fe4ff1c8df None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:26.471616 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf872d2-09bd-47b7-912c-05fe4ff1c8df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:26.487408 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd8f0e8-6de4-4a97-aa53-ed9ce7eeb489 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:43:26.489657 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-cdf872d2-09bd-47b7-912c-05fe4ff1c8df None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:43:26.490874 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-cdf872d2-09bd-47b7-912c-05fe4ff1c8df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:43:26.492446 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e029c4f-5e48-49cb-a5a0-3f059aaf2332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:43:26.493308 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e029c4f-5e48-49cb-a5a0-3f059aaf2332 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:26.493594 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e029c4f-5e48-49cb-a5a0-3f059aaf2332 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:26.493874 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e029c4f-5e48-49cb-a5a0-3f059aaf2332 admin admin] PATH_INFO: `/v3/projects/84eacffbd4ec4b2e9d80c38ab47f02d2` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:26.495146 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e029c4f-5e48-49cb-a5a0-3f059aaf2332 admin admin] RBAC: Authorizing `identity:delete_project(project_id=84eacffbd4ec4b2e9d80c38ab47f02d2)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:43:26.497119 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e029c4f-5e48-49cb-a5a0-3f059aaf2332 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:43:26.499549 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1373/2702] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:43:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 02:43:26.503695 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf872d2-09bd-47b7-912c-05fe4ff1c8df None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:43:26.512587 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf872d2-09bd-47b7-912c-05fe4ff1c8df None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:43:26.518007 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ad3e70-b546-48fd-b00e-27cb11d87037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:43:26.518672 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ad3e70-b546-48fd-b00e-27cb11d87037 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:26.518953 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ad3e70-b546-48fd-b00e-27cb11d87037 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:26.519227 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ad3e70-b546-48fd-b00e-27cb11d87037 admin admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:26.519943 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ad3e70-b546-48fd-b00e-27cb11d87037 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:43:26.520240 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ad3e70-b546-48fd-b00e-27cb11d87037 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:43:26.521806 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ad3e70-b546-48fd-b00e-27cb11d87037 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:43:26.534004 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1374/2703] 10.222.0.66 () {58 vars in 1152 bytes} [Mon Jul 18 02:43:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 02:43:26.538899 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ad3e70-b546-48fd-b00e-27cb11d87037 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:43:26.548605 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ad3e70-b546-48fd-b00e-27cb11d87037 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:43:26.553169 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00c3cb9-b125-453e-a11f-457b6f709804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:43:26.553780 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00c3cb9-b125-453e-a11f-457b6f709804 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:26.554061 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00c3cb9-b125-453e-a11f-457b6f709804 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:26.554364 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00c3cb9-b125-453e-a11f-457b6f709804 admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:26.555028 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00c3cb9-b125-453e-a11f-457b6f709804 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:43:26.556880 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00c3cb9-b125-453e-a11f-457b6f709804 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:43:26.635957 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-3e029c4f-5e48-49cb-a5a0-3f059aaf2332 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84eacffbd4ec4b2e9d80c38ab47f02d2', 'request_id': 'req-3e029c4f-5e48-49cb-a5a0-3f059aaf2332'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:26.642594 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-3e029c4f-5e48-49cb-a5a0-3f059aaf2332 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84eacffbd4ec4b2e9d80c38ab47f02d2', 'request_id': 'req-3e029c4f-5e48-49cb-a5a0-3f059aaf2332'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:26.649120 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-3e029c4f-5e48-49cb-a5a0-3f059aaf2332 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84eacffbd4ec4b2e9d80c38ab47f02d2', 'request_id': 'req-3e029c4f-5e48-49cb-a5a0-3f059aaf2332'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:26.669871 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1375/2704] 10.222.0.66 () {60 vars in 1152 bytes} [Mon Jul 18 02:43:26 2022] POST /identity/v3/projects => generated 375 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 02:43:26.673645 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00c3cb9-b125-453e-a11f-457b6f709804 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:43:26.730750 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:43:26.735205 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00c3cb9-b125-453e-a11f-457b6f709804 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:43:26.739843 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edb740a-91e8-4d45-9bbb-5eccc8343d10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:43:26.740572 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edb740a-91e8-4d45-9bbb-5eccc8343d10 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:26.740853 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edb740a-91e8-4d45-9bbb-5eccc8343d10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:26.741123 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edb740a-91e8-4d45-9bbb-5eccc8343d10 admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:26.741808 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edb740a-91e8-4d45-9bbb-5eccc8343d10 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:43:26.743839 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edb740a-91e8-4d45-9bbb-5eccc8343d10 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:43:26.747139 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-3e029c4f-5e48-49cb-a5a0-3f059aaf2332 admin admin] The token cache is being invalidate because project 84eacffbd4ec4b2e9d80c38ab47f02d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 02:43:26.747435 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-3e029c4f-5e48-49cb-a5a0-3f059aaf2332 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:26.748704 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-3e029c4f-5e48-49cb-a5a0-3f059aaf2332 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:26.750275 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1330/2705] 10.222.0.66 () {58 vars in 1232 bytes} [Mon Jul 18 02:43:26 2022] DELETE /identity/v3/projects/84eacffbd4ec4b2e9d80c38ab47f02d2 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 02:43:26.755238 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25ab865-3f57-4bc7-864f-d899ac85d903 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:26.755526 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25ab865-3f57-4bc7-864f-d899ac85d903 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:26.755804 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25ab865-3f57-4bc7-864f-d899ac85d903 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:26.774966 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-a25ab865-3f57-4bc7-864f-d899ac85d903 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:43:26.776169 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-a25ab865-3f57-4bc7-864f-d899ac85d903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:43:26.784233 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1331/2706] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:43:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 02:43:26.791780 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4db9605-37b6-4635-a285-679170150999 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:26.792071 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4db9605-37b6-4635-a285-679170150999 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:26.792349 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4db9605-37b6-4635-a285-679170150999 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:26.809996 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-a4db9605-37b6-4635-a285-679170150999 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:43:26.811826 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-a4db9605-37b6-4635-a285-679170150999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:43:26.819900 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1332/2707] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:43:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 02:43:26.826530 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1376/2708] 10.222.0.66 () {60 vars in 1143 bytes} [Mon Jul 18 02:43:26 2022] POST /identity/v3/users => generated 288 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 02:43:26.826677 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4db9605-37b6-4635-a285-679170150999 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:43:26.827523 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edb740a-91e8-4d45-9bbb-5eccc8343d10 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:43:26.829157 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-7edb740a-91e8-4d45-9bbb-5eccc8343d10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:43:26.833475 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4db9605-37b6-4635-a285-679170150999 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:43:26.838679 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3505ef3d-990a-43ad-9b64-32b4330c2d32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:43:26.839333 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3505ef3d-990a-43ad-9b64-32b4330c2d32 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:26.839630 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3505ef3d-990a-43ad-9b64-32b4330c2d32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:26.839894 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3505ef3d-990a-43ad-9b64-32b4330c2d32 admin admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:26.840614 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3505ef3d-990a-43ad-9b64-32b4330c2d32 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:43:26.840922 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3505ef3d-990a-43ad-9b64-32b4330c2d32 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:43:26.841771 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edb740a-91e8-4d45-9bbb-5eccc8343d10 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:43:26.842626 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3505ef3d-990a-43ad-9b64-32b4330c2d32 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:43:26.846566 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d42df03-f317-476f-b26d-935de8bf8056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:43:26.847224 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d42df03-f317-476f-b26d-935de8bf8056 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:26.847509 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d42df03-f317-476f-b26d-935de8bf8056 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:26.847783 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d42df03-f317-476f-b26d-935de8bf8056 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:26.848498 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d42df03-f317-476f-b26d-935de8bf8056 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:43:26.850073 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d42df03-f317-476f-b26d-935de8bf8056 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:43:26.855946 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1333/2709] 10.222.0.66 () {58 vars in 1152 bytes} [Mon Jul 18 02:43:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 02:43:26.861347 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1377/2710] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:43:26 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 02:43:26.861486 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3505ef3d-990a-43ad-9b64-32b4330c2d32 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:43:26.864995 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d42df03-f317-476f-b26d-935de8bf8056 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:43:26.871727 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3505ef3d-990a-43ad-9b64-32b4330c2d32 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:43:26.875571 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d42df03-f317-476f-b26d-935de8bf8056 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:43:26.877249 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71365d11-e4ee-439a-8b4a-33d714432258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:43:26.877889 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71365d11-e4ee-439a-8b4a-33d714432258 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:26.878211 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71365d11-e4ee-439a-8b4a-33d714432258 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:26.878485 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71365d11-e4ee-439a-8b4a-33d714432258 admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:26.879170 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71365d11-e4ee-439a-8b4a-33d714432258 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:43:26.880182 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f37cfa6-4f52-4db0-afd3-11f959b55144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:43:26.880848 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f37cfa6-4f52-4db0-afd3-11f959b55144 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:26.881061 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71365d11-e4ee-439a-8b4a-33d714432258 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:43:26.881173 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f37cfa6-4f52-4db0-afd3-11f959b55144 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:26.881399 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f37cfa6-4f52-4db0-afd3-11f959b55144 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:26.882127 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f37cfa6-4f52-4db0-afd3-11f959b55144 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:43:26.883711 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f37cfa6-4f52-4db0-afd3-11f959b55144 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:43:26.885267 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:43:26.896605 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1378/2711] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:43:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 02:43:26.901625 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f37cfa6-4f52-4db0-afd3-11f959b55144 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:43:26.911674 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f37cfa6-4f52-4db0-afd3-11f959b55144 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:43:26.916250 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d703f88-20fd-4d65-b46b-961ab2cc9367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:43:26.917125 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d703f88-20fd-4d65-b46b-961ab2cc9367 admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:26.917393 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d703f88-20fd-4d65-b46b-961ab2cc9367 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:26.917665 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d703f88-20fd-4d65-b46b-961ab2cc9367 admin admin] PATH_INFO: `/v3/projects/7fc721738d474d0fa344aa5e80f66869/users/201a447a65a54dcb8bf85cc1cbb17d9c/roles/65feabf3b27944a8b01a86c8ac608457` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:26.934927 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d703f88-20fd-4d65-b46b-961ab2cc9367 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7fc721738d474d0fa344aa5e80f66869, user_id=201a447a65a54dcb8bf85cc1cbb17d9c, role_id=65feabf3b27944a8b01a86c8ac608457)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:43:26.938836 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d703f88-20fd-4d65-b46b-961ab2cc9367 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:43:27.014709 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1334/2712] 10.222.0.66 () {60 vars in 1152 bytes} [Mon Jul 18 02:43:26 2022] POST /identity/v3/projects => generated 399 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 02:43:27.018632 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71365d11-e4ee-439a-8b4a-33d714432258 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:43:27.040594 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1379/2713] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:43:26 2022] PUT /identity/v3/projects/7fc721738d474d0fa344aa5e80f66869/users/201a447a65a54dcb8bf85cc1cbb17d9c/roles/65feabf3b27944a8b01a86c8ac608457 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 02:43:27.053410 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f03d1d4-d20a-4560-8e60-06c3e9a01100 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:27.053724 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f03d1d4-d20a-4560-8e60-06c3e9a01100 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:27.054026 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f03d1d4-d20a-4560-8e60-06c3e9a01100 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:27.081338 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-7f03d1d4-d20a-4560-8e60-06c3e9a01100 None None] MFA Rules not processed for user `201a447a65a54dcb8bf85cc1cbb17d9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:43:27.082953 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-7f03d1d4-d20a-4560-8e60-06c3e9a01100 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:43:27.084373 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71365d11-e4ee-439a-8b4a-33d714432258 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:43:27.146037 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ccf4d6-4053-4a5c-b430-c59995d82dd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:43:27.146782 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ccf4d6-4053-4a5c-b430-c59995d82dd9 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:27.147018 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ccf4d6-4053-4a5c-b430-c59995d82dd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:27.147293 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ccf4d6-4053-4a5c-b430-c59995d82dd9 admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:27.147982 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ccf4d6-4053-4a5c-b430-c59995d82dd9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:43:27.149921 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ccf4d6-4053-4a5c-b430-c59995d82dd9 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:43:27.151797 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1380/2714] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:43:27 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 02:43:27.166311 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f03d1d4-d20a-4560-8e60-06c3e9a01100 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:43:27.167656 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-7f03d1d4-d20a-4560-8e60-06c3e9a01100 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:43:27.226911 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f03d1d4-d20a-4560-8e60-06c3e9a01100 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:43:27.230955 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c80ac9e-da41-4580-97ac-a46baefbdb45 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1391b81e7234bc18e22b9e84f231256', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:43:27.231695 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c80ac9e-da41-4580-97ac-a46baefbdb45 service glance] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:27.232627 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c80ac9e-da41-4580-97ac-a46baefbdb45 service glance] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:27.232928 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c80ac9e-da41-4580-97ac-a46baefbdb45 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:27.241990 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c80ac9e-da41-4580-97ac-a46baefbdb45 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:43:27.243507 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c80ac9e-da41-4580-97ac-a46baefbdb45 service glance] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:43:27.245028 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1335/2715] 10.222.0.66 () {60 vars in 1143 bytes} [Mon Jul 18 02:43:27 2022] POST /identity/v3/users => generated 300 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 02:43:27.249164 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ccf4d6-4053-4a5c-b430-c59995d82dd9 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:43:27.253843 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1381/2716] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:43:27 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 02:43:27.259065 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ccf4d6-4053-4a5c-b430-c59995d82dd9 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:43:27.263801 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20c89ef-2b2e-4320-aa40-553938ab9104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:43:27.264485 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20c89ef-2b2e-4320-aa40-553938ab9104 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:27.264829 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20c89ef-2b2e-4320-aa40-553938ab9104 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:27.265099 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20c89ef-2b2e-4320-aa40-553938ab9104 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:27.266056 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20c89ef-2b2e-4320-aa40-553938ab9104 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:43:27.268020 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20c89ef-2b2e-4320-aa40-553938ab9104 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:43:27.289342 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1336/2717] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:43:27 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 02:43:27.292601 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c80ac9e-da41-4580-97ac-a46baefbdb45 service glance] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:43:27.305373 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c80ac9e-da41-4580-97ac-a46baefbdb45 service glance] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:43:27.311036 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfec41a-0134-44ee-b23f-d8ff2c40c745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:43:27.311924 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfec41a-0134-44ee-b23f-d8ff2c40c745 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:27.312387 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfec41a-0134-44ee-b23f-d8ff2c40c745 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:27.312729 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfec41a-0134-44ee-b23f-d8ff2c40c745 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:27.313929 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfec41a-0134-44ee-b23f-d8ff2c40c745 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:43:27.315709 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfec41a-0134-44ee-b23f-d8ff2c40c745 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:43:27.328254 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1382/2718] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:43:27 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:43:27.332258 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20c89ef-2b2e-4320-aa40-553938ab9104 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:43:27.343109 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20c89ef-2b2e-4320-aa40-553938ab9104 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:43:27.348761 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366eb993-d895-4fb7-bc39-6f8ff3843368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:43:27.348955 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366eb993-d895-4fb7-bc39-6f8ff3843368 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:27.349152 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366eb993-d895-4fb7-bc39-6f8ff3843368 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:27.349429 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366eb993-d895-4fb7-bc39-6f8ff3843368 admin admin] PATH_INFO: `/v3/projects/5199dc4436db4ef480604ddb244a1f4a/users/9b9d72e248a340769ddf5c09ef555ad0/roles/65feabf3b27944a8b01a86c8ac608457` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:27.365571 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366eb993-d895-4fb7-bc39-6f8ff3843368 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5199dc4436db4ef480604ddb244a1f4a, user_id=9b9d72e248a340769ddf5c09ef555ad0, role_id=65feabf3b27944a8b01a86c8ac608457)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:43:27.368280 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366eb993-d895-4fb7-bc39-6f8ff3843368 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:43:27.441658 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1337/2719] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:43:27 2022] PUT /identity/v3/projects/5199dc4436db4ef480604ddb244a1f4a/users/9b9d72e248a340769ddf5c09ef555ad0/roles/65feabf3b27944a8b01a86c8ac608457 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:43:27.509028 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ab48cf-6f30-400c-a0af-0a0f5b77227f None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:27.509602 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ab48cf-6f30-400c-a0af-0a0f5b77227f None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:27.509602 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ab48cf-6f30-400c-a0af-0a0f5b77227f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:27.538609 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-33ab48cf-6f30-400c-a0af-0a0f5b77227f None None] MFA Rules not processed for user `9b9d72e248a340769ddf5c09ef555ad0`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:43:27.539847 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-33ab48cf-6f30-400c-a0af-0a0f5b77227f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:43:27.551291 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:43:27.608506 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1383/2720] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:43:27 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:43:27.619867 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366eb993-d895-4fb7-bc39-6f8ff3843368 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:43:27.621552 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-366eb993-d895-4fb7-bc39-6f8ff3843368 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:43:27.679654 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:43:27.705661 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366eb993-d895-4fb7-bc39-6f8ff3843368 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:43:27.709496 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5043e4c-3d25-4978-833c-ef0344f835ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd49388facdf446885e55b45ef91f01e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:43:27.710207 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5043e4c-3d25-4978-833c-ef0344f835ff service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:27.710482 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5043e4c-3d25-4978-833c-ef0344f835ff service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:27.710762 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5043e4c-3d25-4978-833c-ef0344f835ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:27.722620 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5043e4c-3d25-4978-833c-ef0344f835ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:43:27.724170 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5043e4c-3d25-4978-833c-ef0344f835ff service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:43:27.733895 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1338/2721] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:43:27 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:43:36.565764 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ab48cf-6f30-400c-a0af-0a0f5b77227f None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:43:36.567321 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-33ab48cf-6f30-400c-a0af-0a0f5b77227f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:43:36.637119 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ab48cf-6f30-400c-a0af-0a0f5b77227f None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:43:36.641982 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f684cff-83ff-429a-9a27-cd08bea29c05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:43:36.642870 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f684cff-83ff-429a-9a27-cd08bea29c05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:36.643155 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f684cff-83ff-429a-9a27-cd08bea29c05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:36.643457 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f684cff-83ff-429a-9a27-cd08bea29c05 admin admin] PATH_INFO: `/v3/users/2715041954204ec4bd55d5b5719e9bd8` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:36.644719 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f684cff-83ff-429a-9a27-cd08bea29c05 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2715041954204ec4bd55d5b5719e9bd8)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:43:36.646725 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f684cff-83ff-429a-9a27-cd08bea29c05 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:43:36.903369 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-6f684cff-83ff-429a-9a27-cd08bea29c05 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2715041954204ec4bd55d5b5719e9bd8'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:36.989304 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5043e4c-3d25-4978-833c-ef0344f835ff service cinder] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:43:37.000386 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5043e4c-3d25-4978-833c-ef0344f835ff service cinder] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:43:37.004863 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824d8213-5820-498f-b4b9-17fa838bf0a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:43:37.005724 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824d8213-5820-498f-b4b9-17fa838bf0a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:37.006003 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824d8213-5820-498f-b4b9-17fa838bf0a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:37.006310 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824d8213-5820-498f-b4b9-17fa838bf0a5 admin admin] PATH_INFO: `/v3/users/9b9d72e248a340769ddf5c09ef555ad0` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:37.007479 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824d8213-5820-498f-b4b9-17fa838bf0a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b9d72e248a340769ddf5c09ef555ad0)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:43:37.009377 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824d8213-5820-498f-b4b9-17fa838bf0a5 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:43:37.075023 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-6f684cff-83ff-429a-9a27-cd08bea29c05 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2715041954204ec4bd55d5b5719e9bd8'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:37.082781 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-6f684cff-83ff-429a-9a27-cd08bea29c05 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2715041954204ec4bd55d5b5719e9bd8'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:37.083492 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-6f684cff-83ff-429a-9a27-cd08bea29c05 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2715041954204ec4bd55d5b5719e9bd8'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:37.096412 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-6f684cff-83ff-429a-9a27-cd08bea29c05 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2715041954204ec4bd55d5b5719e9bd8'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:37.102884 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1384/2722] 10.222.0.66 () {58 vars in 1223 bytes} [Mon Jul 18 02:43:36 2022] DELETE /identity/v3/users/2715041954204ec4bd55d5b5719e9bd8 => generated 0 bytes in 538 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 02:43:37.107928 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad59dfe-c9eb-4be8-9a55-1f8354ea9e9b None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:37.108223 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad59dfe-c9eb-4be8-9a55-1f8354ea9e9b None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:37.108502 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad59dfe-c9eb-4be8-9a55-1f8354ea9e9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:37.126194 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-0ad59dfe-c9eb-4be8-9a55-1f8354ea9e9b None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:43:37.129243 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-0ad59dfe-c9eb-4be8-9a55-1f8354ea9e9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:43:37.155956 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:43:37.189990 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1385/2723] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:43:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:43:37.200042 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad59dfe-c9eb-4be8-9a55-1f8354ea9e9b None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:43:37.201935 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-0ad59dfe-c9eb-4be8-9a55-1f8354ea9e9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:43:37.261679 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad59dfe-c9eb-4be8-9a55-1f8354ea9e9b None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:43:37.267962 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9948c9-9ecd-46ad-9e17-3e3336a1d7d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eaf19a259c2741cea375a38763e33cdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:43:37.268366 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-824d8213-5820-498f-b4b9-17fa838bf0a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b9d72e248a340769ddf5c09ef555ad0'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:37.268520 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9948c9-9ecd-46ad-9e17-3e3336a1d7d2 service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:37.268790 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9948c9-9ecd-46ad-9e17-3e3336a1d7d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:37.269063 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9948c9-9ecd-46ad-9e17-3e3336a1d7d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:37.278944 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9948c9-9ecd-46ad-9e17-3e3336a1d7d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:43:37.280388 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9948c9-9ecd-46ad-9e17-3e3336a1d7d2 service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:43:37.290057 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1386/2724] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:43:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 02:43:37.383281 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-824d8213-5820-498f-b4b9-17fa838bf0a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b9d72e248a340769ddf5c09ef555ad0'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:37.383950 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-824d8213-5820-498f-b4b9-17fa838bf0a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b9d72e248a340769ddf5c09ef555ad0'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:37.396982 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-824d8213-5820-498f-b4b9-17fa838bf0a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b9d72e248a340769ddf5c09ef555ad0'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:37.397657 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-824d8213-5820-498f-b4b9-17fa838bf0a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b9d72e248a340769ddf5c09ef555ad0'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:37.410697 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1339/2725] 10.222.0.66 () {58 vars in 1223 bytes} [Mon Jul 18 02:43:36 2022] DELETE /identity/v3/users/9b9d72e248a340769ddf5c09ef555ad0 => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 02:43:37.415139 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eed5b0c-2180-4ef7-9a6f-5104e811ae0f None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:37.415450 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eed5b0c-2180-4ef7-9a6f-5104e811ae0f None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:37.417195 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eed5b0c-2180-4ef7-9a6f-5104e811ae0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:37.434664 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-9eed5b0c-2180-4ef7-9a6f-5104e811ae0f None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:43:37.436528 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-9eed5b0c-2180-4ef7-9a6f-5104e811ae0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:43:37.515344 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1387/2726] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:43:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:43:37.530875 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824d8213-5820-498f-b4b9-17fa838bf0a5 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:43:37.535788 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-824d8213-5820-498f-b4b9-17fa838bf0a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:43:37.607425 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824d8213-5820-498f-b4b9-17fa838bf0a5 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:43:37.611337 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d04e41-3582-4d66-b1e8-70dd406c25af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eaf19a259c2741cea375a38763e33cdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:43:37.612234 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d04e41-3582-4d66-b1e8-70dd406c25af service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:37.612574 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d04e41-3582-4d66-b1e8-70dd406c25af service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:37.612932 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d04e41-3582-4d66-b1e8-70dd406c25af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:37.626804 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d04e41-3582-4d66-b1e8-70dd406c25af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:43:37.628261 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d04e41-3582-4d66-b1e8-70dd406c25af service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:43:37.639148 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1340/2727] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:43:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 02:43:37.750993 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eed5b0c-2180-4ef7-9a6f-5104e811ae0f None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:43:37.752712 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-9eed5b0c-2180-4ef7-9a6f-5104e811ae0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:43:37.771069 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eed5b0c-2180-4ef7-9a6f-5104e811ae0f None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:43:37.777335 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890293f3-c8b1-486b-87f1-431a73267e5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:43:37.781440 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890293f3-c8b1-486b-87f1-431a73267e5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:37.781987 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890293f3-c8b1-486b-87f1-431a73267e5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:37.783127 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890293f3-c8b1-486b-87f1-431a73267e5e admin admin] PATH_INFO: `/v3/projects/e8200538fea84e46b424ede0d9b719d8` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:37.784509 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890293f3-c8b1-486b-87f1-431a73267e5e admin admin] RBAC: Authorizing `identity:delete_project(project_id=e8200538fea84e46b424ede0d9b719d8)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:43:37.786645 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890293f3-c8b1-486b-87f1-431a73267e5e admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:43:37.915909 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-890293f3-c8b1-486b-87f1-431a73267e5e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8200538fea84e46b424ede0d9b719d8', 'request_id': 'req-890293f3-c8b1-486b-87f1-431a73267e5e'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:37.922708 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-890293f3-c8b1-486b-87f1-431a73267e5e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8200538fea84e46b424ede0d9b719d8', 'request_id': 'req-890293f3-c8b1-486b-87f1-431a73267e5e'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:38.011711 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-890293f3-c8b1-486b-87f1-431a73267e5e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8200538fea84e46b424ede0d9b719d8', 'request_id': 'req-890293f3-c8b1-486b-87f1-431a73267e5e'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:38.020299 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:43:38.025538 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-890293f3-c8b1-486b-87f1-431a73267e5e admin admin] The token cache is being invalidate because project e8200538fea84e46b424ede0d9b719d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 02:43:38.025887 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-890293f3-c8b1-486b-87f1-431a73267e5e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:38.026648 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-890293f3-c8b1-486b-87f1-431a73267e5e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:38.032051 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1388/2728] 10.222.0.66 () {58 vars in 1232 bytes} [Mon Jul 18 02:43:37 2022] DELETE /identity/v3/projects/e8200538fea84e46b424ede0d9b719d8 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:43:38.033557 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d04e41-3582-4d66-b1e8-70dd406c25af service neutron] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:43:38.043436 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d04e41-3582-4d66-b1e8-70dd406c25af service neutron] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:43:38.089646 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:43:38.104178 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e561dcb-03f4-49d7-8da5-cee241f2f626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:43:38.105244 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e561dcb-03f4-49d7-8da5-cee241f2f626 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:38.105544 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e561dcb-03f4-49d7-8da5-cee241f2f626 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:38.105805 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e561dcb-03f4-49d7-8da5-cee241f2f626 admin admin] PATH_INFO: `/v3/users/9b4c027b17bf42e4a6b49b9394fd64dc` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:38.107486 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e561dcb-03f4-49d7-8da5-cee241f2f626 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b4c027b17bf42e4a6b49b9394fd64dc)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:43:38.109647 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e561dcb-03f4-49d7-8da5-cee241f2f626 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:43:38.271333 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890293f3-c8b1-486b-87f1-431a73267e5e admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:43:38.272857 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-890293f3-c8b1-486b-87f1-431a73267e5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:43:38.287484 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890293f3-c8b1-486b-87f1-431a73267e5e admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:43:38.292194 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0331f6a5-9f93-4281-9e3c-bff3ea2483d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:43:38.294511 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0331f6a5-9f93-4281-9e3c-bff3ea2483d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:38.294511 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0331f6a5-9f93-4281-9e3c-bff3ea2483d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:38.294511 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0331f6a5-9f93-4281-9e3c-bff3ea2483d9 admin admin] PATH_INFO: `/v3/projects/5199dc4436db4ef480604ddb244a1f4a` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:38.295015 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0331f6a5-9f93-4281-9e3c-bff3ea2483d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5199dc4436db4ef480604ddb244a1f4a)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:43:38.297004 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0331f6a5-9f93-4281-9e3c-bff3ea2483d9 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:43:38.379541 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-8e561dcb-03f4-49d7-8da5-cee241f2f626 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b4c027b17bf42e4a6b49b9394fd64dc'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:38.458652 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-0331f6a5-9f93-4281-9e3c-bff3ea2483d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5199dc4436db4ef480604ddb244a1f4a', 'request_id': 'req-0331f6a5-9f93-4281-9e3c-bff3ea2483d9'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:38.464323 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-0331f6a5-9f93-4281-9e3c-bff3ea2483d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5199dc4436db4ef480604ddb244a1f4a', 'request_id': 'req-0331f6a5-9f93-4281-9e3c-bff3ea2483d9'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:38.481139 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-8e561dcb-03f4-49d7-8da5-cee241f2f626 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b4c027b17bf42e4a6b49b9394fd64dc'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:38.481880 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-8e561dcb-03f4-49d7-8da5-cee241f2f626 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b4c027b17bf42e4a6b49b9394fd64dc'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:38.494863 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-8e561dcb-03f4-49d7-8da5-cee241f2f626 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b4c027b17bf42e4a6b49b9394fd64dc'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:38.495520 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-8e561dcb-03f4-49d7-8da5-cee241f2f626 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b4c027b17bf42e4a6b49b9394fd64dc'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:38.508435 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1341/2729] 10.222.0.66 () {58 vars in 1223 bytes} [Mon Jul 18 02:43:38 2022] DELETE /identity/v3/users/9b4c027b17bf42e4a6b49b9394fd64dc => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:43:38.578224 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-0331f6a5-9f93-4281-9e3c-bff3ea2483d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5199dc4436db4ef480604ddb244a1f4a', 'request_id': 'req-0331f6a5-9f93-4281-9e3c-bff3ea2483d9'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:38.590035 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-0331f6a5-9f93-4281-9e3c-bff3ea2483d9 admin admin] The token cache is being invalidate because project 5199dc4436db4ef480604ddb244a1f4a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 02:43:38.590355 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-0331f6a5-9f93-4281-9e3c-bff3ea2483d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:38.590997 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-0331f6a5-9f93-4281-9e3c-bff3ea2483d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:38.593001 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1389/2730] 10.222.0.66 () {58 vars in 1232 bytes} [Mon Jul 18 02:43:38 2022] DELETE /identity/v3/projects/5199dc4436db4ef480604ddb244a1f4a => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 02:43:39.002391 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e561dcb-03f4-49d7-8da5-cee241f2f626 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:43:39.003963 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-8e561dcb-03f4-49d7-8da5-cee241f2f626 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:43:39.074837 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e561dcb-03f4-49d7-8da5-cee241f2f626 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:43:39.079505 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fed2ec-43a8-494f-a6ff-06fd85058b37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:43:39.080370 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fed2ec-43a8-494f-a6ff-06fd85058b37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:39.080635 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fed2ec-43a8-494f-a6ff-06fd85058b37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:39.080906 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fed2ec-43a8-494f-a6ff-06fd85058b37 admin admin] PATH_INFO: `/v3/projects/05b150a2fea54fd8895ae364690e28e0` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:39.082077 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fed2ec-43a8-494f-a6ff-06fd85058b37 admin admin] RBAC: Authorizing `identity:delete_project(project_id=05b150a2fea54fd8895ae364690e28e0)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:43:39.084007 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fed2ec-43a8-494f-a6ff-06fd85058b37 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:43:39.210715 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-99fed2ec-43a8-494f-a6ff-06fd85058b37 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05b150a2fea54fd8895ae364690e28e0', 'request_id': 'req-99fed2ec-43a8-494f-a6ff-06fd85058b37'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:39.216656 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-99fed2ec-43a8-494f-a6ff-06fd85058b37 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05b150a2fea54fd8895ae364690e28e0', 'request_id': 'req-99fed2ec-43a8-494f-a6ff-06fd85058b37'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:39.222884 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-99fed2ec-43a8-494f-a6ff-06fd85058b37 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05b150a2fea54fd8895ae364690e28e0', 'request_id': 'req-99fed2ec-43a8-494f-a6ff-06fd85058b37'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:39.226676 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:43:39.320743 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-99fed2ec-43a8-494f-a6ff-06fd85058b37 admin admin] The token cache is being invalidate because project 05b150a2fea54fd8895ae364690e28e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 02:43:39.321019 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-99fed2ec-43a8-494f-a6ff-06fd85058b37 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:39.321652 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-99fed2ec-43a8-494f-a6ff-06fd85058b37 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:39.323227 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1342/2731] 10.222.0.66 () {58 vars in 1232 bytes} [Mon Jul 18 02:43:39 2022] DELETE /identity/v3/projects/05b150a2fea54fd8895ae364690e28e0 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:43:39.328541 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6147030c-299c-4b9b-bbbd-076985d3ac46 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:39.329033 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6147030c-299c-4b9b-bbbd-076985d3ac46 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:39.329033 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6147030c-299c-4b9b-bbbd-076985d3ac46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:39.348856 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-6147030c-299c-4b9b-bbbd-076985d3ac46 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:43:39.350194 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-6147030c-299c-4b9b-bbbd-076985d3ac46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:43:39.413183 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1390/2732] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:43:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:43:39.422333 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02a7101-4b94-49eb-aa92-1ab5db77b8de None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:39.422631 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02a7101-4b94-49eb-aa92-1ab5db77b8de None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:39.422912 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02a7101-4b94-49eb-aa92-1ab5db77b8de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:39.442142 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-b02a7101-4b94-49eb-aa92-1ab5db77b8de None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:43:39.443370 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-b02a7101-4b94-49eb-aa92-1ab5db77b8de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:43:39.451868 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1343/2733] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:43:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:43:39.456534 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6147030c-299c-4b9b-bbbd-076985d3ac46 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:43:39.467396 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6147030c-299c-4b9b-bbbd-076985d3ac46 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:43:39.472079 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d01e79-5dbe-4064-b2d8-3f3da4ea9a91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:43:39.472747 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d01e79-5dbe-4064-b2d8-3f3da4ea9a91 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:39.473035 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d01e79-5dbe-4064-b2d8-3f3da4ea9a91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:39.473320 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d01e79-5dbe-4064-b2d8-3f3da4ea9a91 admin admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:39.474067 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d01e79-5dbe-4064-b2d8-3f3da4ea9a91 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:43:39.474467 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d01e79-5dbe-4064-b2d8-3f3da4ea9a91 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:43:39.476168 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d01e79-5dbe-4064-b2d8-3f3da4ea9a91 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:43:39.489183 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1391/2734] 10.222.0.66 () {58 vars in 1152 bytes} [Mon Jul 18 02:43:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:43:39.495798 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02a7101-4b94-49eb-aa92-1ab5db77b8de None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:43:39.505901 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02a7101-4b94-49eb-aa92-1ab5db77b8de None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:43:39.510711 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0e4616-fe5a-40bf-9398-751a89fc3473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:43:39.511358 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0e4616-fe5a-40bf-9398-751a89fc3473 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:39.511641 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0e4616-fe5a-40bf-9398-751a89fc3473 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:39.511915 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0e4616-fe5a-40bf-9398-751a89fc3473 admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:39.512605 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0e4616-fe5a-40bf-9398-751a89fc3473 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:43:39.515146 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0e4616-fe5a-40bf-9398-751a89fc3473 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:43:39.612155 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1344/2735] 10.222.0.66 () {60 vars in 1152 bytes} [Mon Jul 18 02:43:39 2022] POST /identity/v3/projects => generated 387 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:43:39.613718 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d01e79-5dbe-4064-b2d8-3f3da4ea9a91 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:43:39.624626 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d01e79-5dbe-4064-b2d8-3f3da4ea9a91 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:43:39.687224 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937b5816-4336-4a8d-93ba-1138bdc26582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:43:39.687890 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937b5816-4336-4a8d-93ba-1138bdc26582 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:39.688259 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937b5816-4336-4a8d-93ba-1138bdc26582 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:39.688584 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937b5816-4336-4a8d-93ba-1138bdc26582 admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:39.689608 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937b5816-4336-4a8d-93ba-1138bdc26582 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:43:39.691808 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937b5816-4336-4a8d-93ba-1138bdc26582 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:43:39.767014 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:43:39.773222 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1392/2736] 10.222.0.66 () {60 vars in 1143 bytes} [Mon Jul 18 02:43:39 2022] POST /identity/v3/users => generated 294 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:43:39.777134 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0e4616-fe5a-40bf-9398-751a89fc3473 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:43:39.788831 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0e4616-fe5a-40bf-9398-751a89fc3473 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:43:39.793434 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c998465f-f407-4303-8c3c-07a436976b87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:43:39.794180 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c998465f-f407-4303-8c3c-07a436976b87 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:39.794706 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c998465f-f407-4303-8c3c-07a436976b87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:39.794964 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c998465f-f407-4303-8c3c-07a436976b87 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:39.795685 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c998465f-f407-4303-8c3c-07a436976b87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:43:39.797293 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c998465f-f407-4303-8c3c-07a436976b87 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:43:39.808565 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1345/2737] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:43:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:43:39.813475 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937b5816-4336-4a8d-93ba-1138bdc26582 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:43:39.824236 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937b5816-4336-4a8d-93ba-1138bdc26582 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:43:39.829668 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57572240-2aa2-4c0e-878b-81fefaf1f75b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:43:39.830372 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57572240-2aa2-4c0e-878b-81fefaf1f75b admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:39.830689 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57572240-2aa2-4c0e-878b-81fefaf1f75b admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:39.830995 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57572240-2aa2-4c0e-878b-81fefaf1f75b admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:39.831722 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57572240-2aa2-4c0e-878b-81fefaf1f75b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:43:39.833585 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57572240-2aa2-4c0e-878b-81fefaf1f75b admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:43:39.844241 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1393/2738] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:43:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:43:39.848688 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c998465f-f407-4303-8c3c-07a436976b87 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:43:39.859126 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c998465f-f407-4303-8c3c-07a436976b87 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:43:39.864526 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c4e490-ab48-4cf4-a592-de3805b4feae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:43:39.865418 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c4e490-ab48-4cf4-a592-de3805b4feae admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:39.865698 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c4e490-ab48-4cf4-a592-de3805b4feae admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:39.865974 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c4e490-ab48-4cf4-a592-de3805b4feae admin admin] PATH_INFO: `/v3/projects/49e6c9938c6145db9c4af1094aae8985/users/51bbb75a20334a2ca4b87970aeb06945/roles/65feabf3b27944a8b01a86c8ac608457` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:39.881868 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c4e490-ab48-4cf4-a592-de3805b4feae admin admin] RBAC: Authorizing `identity:create_grant(project_id=49e6c9938c6145db9c4af1094aae8985, user_id=51bbb75a20334a2ca4b87970aeb06945, role_id=65feabf3b27944a8b01a86c8ac608457)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:43:39.884676 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c4e490-ab48-4cf4-a592-de3805b4feae admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:43:39.967861 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1346/2739] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:43:39 2022] PUT /identity/v3/projects/49e6c9938c6145db9c4af1094aae8985/users/51bbb75a20334a2ca4b87970aeb06945/roles/65feabf3b27944a8b01a86c8ac608457 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 02:43:39.978378 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a568b624-9d16-46d6-9c11-5e4b97465d83 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:39.978696 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a568b624-9d16-46d6-9c11-5e4b97465d83 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:39.978974 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a568b624-9d16-46d6-9c11-5e4b97465d83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:40.012776 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-a568b624-9d16-46d6-9c11-5e4b97465d83 None None] MFA Rules not processed for user `51bbb75a20334a2ca4b87970aeb06945`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:43:40.014068 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-a568b624-9d16-46d6-9c11-5e4b97465d83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:43:40.087323 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1394/2740] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:43:39 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:43:40.092364 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c4e490-ab48-4cf4-a592-de3805b4feae admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:43:40.159297 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c4e490-ab48-4cf4-a592-de3805b4feae admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:43:40.163757 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db019f02-44fd-48e8-9039-b6b92eb229d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:43:40.164540 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db019f02-44fd-48e8-9039-b6b92eb229d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:40.164832 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db019f02-44fd-48e8-9039-b6b92eb229d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:40.165109 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db019f02-44fd-48e8-9039-b6b92eb229d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:40.165826 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db019f02-44fd-48e8-9039-b6b92eb229d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:43:40.167786 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db019f02-44fd-48e8-9039-b6b92eb229d1 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:43:40.248964 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1347/2741] 10.222.0.66 () {60 vars in 1152 bytes} [Mon Jul 18 02:43:40 2022] POST /identity/v3/projects => generated 385 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:43:40.251695 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a568b624-9d16-46d6-9c11-5e4b97465d83 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:43:40.264422 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a568b624-9d16-46d6-9c11-5e4b97465d83 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:43:40.324125 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ed9fd0-5519-4a69-afb3-6e457b56e794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:43:40.324803 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ed9fd0-5519-4a69-afb3-6e457b56e794 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:40.325089 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ed9fd0-5519-4a69-afb3-6e457b56e794 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:40.325878 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ed9fd0-5519-4a69-afb3-6e457b56e794 admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:40.326625 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ed9fd0-5519-4a69-afb3-6e457b56e794 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:43:40.328558 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ed9fd0-5519-4a69-afb3-6e457b56e794 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:43:40.461898 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1395/2742] 10.222.0.66 () {60 vars in 1143 bytes} [Mon Jul 18 02:43:40 2022] POST /identity/v3/users => generated 299 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:43:40.463381 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db019f02-44fd-48e8-9039-b6b92eb229d1 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:43:40.475243 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db019f02-44fd-48e8-9039-b6b92eb229d1 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:43:40.480095 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3b3e0b-abf9-4f67-b050-7563e1b921a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:43:40.480773 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3b3e0b-abf9-4f67-b050-7563e1b921a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:40.481056 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3b3e0b-abf9-4f67-b050-7563e1b921a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:40.481333 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3b3e0b-abf9-4f67-b050-7563e1b921a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:40.482610 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3b3e0b-abf9-4f67-b050-7563e1b921a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:43:40.484426 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3b3e0b-abf9-4f67-b050-7563e1b921a4 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:43:40.496514 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1348/2743] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:43:40 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 02:43:40.501855 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ed9fd0-5519-4a69-afb3-6e457b56e794 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:43:40.512375 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ed9fd0-5519-4a69-afb3-6e457b56e794 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:43:40.517568 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b99a06-2354-4836-94f4-ee39ccd45234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:43:40.518488 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b99a06-2354-4836-94f4-ee39ccd45234 admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:40.518769 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b99a06-2354-4836-94f4-ee39ccd45234 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:40.519076 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b99a06-2354-4836-94f4-ee39ccd45234 admin admin] PATH_INFO: `/v3/projects/46406bc0cf33427c9714556c833de65d/users/4e7cb443b0f743ce96a9f9bd866637d5/roles/d4aa3324f0c84887b41d0dc4acf7f30e` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:40.538663 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b99a06-2354-4836-94f4-ee39ccd45234 admin admin] RBAC: Authorizing `identity:create_grant(project_id=46406bc0cf33427c9714556c833de65d, user_id=4e7cb443b0f743ce96a9f9bd866637d5, role_id=d4aa3324f0c84887b41d0dc4acf7f30e)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:43:40.541479 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b99a06-2354-4836-94f4-ee39ccd45234 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:43:40.619670 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1396/2744] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:43:40 2022] PUT /identity/v3/projects/46406bc0cf33427c9714556c833de65d/users/4e7cb443b0f743ce96a9f9bd866637d5/roles/d4aa3324f0c84887b41d0dc4acf7f30e => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:43:40.624479 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3b3e0b-abf9-4f67-b050-7563e1b921a4 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:43:40.634667 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3b3e0b-abf9-4f67-b050-7563e1b921a4 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:43:40.695858 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81051c11-b773-4c3c-9897-650697719d65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:43:40.696532 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81051c11-b773-4c3c-9897-650697719d65 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:40.701584 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81051c11-b773-4c3c-9897-650697719d65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:40.701584 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81051c11-b773-4c3c-9897-650697719d65 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:40.701584 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81051c11-b773-4c3c-9897-650697719d65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:43:40.701584 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81051c11-b773-4c3c-9897-650697719d65 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:43:40.711308 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1349/2745] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:43:40 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:43:40.715402 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b99a06-2354-4836-94f4-ee39ccd45234 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:43:40.725857 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b99a06-2354-4836-94f4-ee39ccd45234 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:43:40.730601 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02220f4a-e80e-4db0-ba61-96d119f53ace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:43:40.731502 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02220f4a-e80e-4db0-ba61-96d119f53ace admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:40.731921 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02220f4a-e80e-4db0-ba61-96d119f53ace admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:40.732158 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02220f4a-e80e-4db0-ba61-96d119f53ace admin admin] PATH_INFO: `/v3/projects/46406bc0cf33427c9714556c833de65d/users/4e7cb443b0f743ce96a9f9bd866637d5/roles/d4aa3324f0c84887b41d0dc4acf7f30e` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:40.734218 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02220f4a-e80e-4db0-ba61-96d119f53ace admin admin] RBAC: Authorizing `identity:create_grant(project_id=46406bc0cf33427c9714556c833de65d, user_id=4e7cb443b0f743ce96a9f9bd866637d5, role_id=d4aa3324f0c84887b41d0dc4acf7f30e)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:43:40.736907 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02220f4a-e80e-4db0-ba61-96d119f53ace admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:43:40.748115 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1397/2746] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:43:40 2022] PUT /identity/v3/projects/46406bc0cf33427c9714556c833de65d/users/4e7cb443b0f743ce96a9f9bd866637d5/roles/d4aa3324f0c84887b41d0dc4acf7f30e => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:43:40.758636 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a916b6ec-7ab4-4fc8-ad74-881a6ed21af8 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:40.758949 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a916b6ec-7ab4-4fc8-ad74-881a6ed21af8 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:40.759226 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a916b6ec-7ab4-4fc8-ad74-881a6ed21af8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:40.765722 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:43:40.788140 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-a916b6ec-7ab4-4fc8-ad74-881a6ed21af8 None None] MFA Rules not processed for user `4e7cb443b0f743ce96a9f9bd866637d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:43:40.789405 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-a916b6ec-7ab4-4fc8-ad74-881a6ed21af8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:43:40.862483 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1350/2747] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:43:40 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 02:43:40.874869 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02220f4a-e80e-4db0-ba61-96d119f53ace admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:43:40.879037 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-02220f4a-e80e-4db0-ba61-96d119f53ace admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:43:40.912202 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:43:40.940696 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02220f4a-e80e-4db0-ba61-96d119f53ace admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:43:40.944527 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8cbeb8-4169-4d46-a1f9-e827d944a1a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd49388facdf446885e55b45ef91f01e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:43:40.947690 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8cbeb8-4169-4d46-a1f9-e827d944a1a0 service cinder] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:40.947690 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8cbeb8-4169-4d46-a1f9-e827d944a1a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:40.947690 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8cbeb8-4169-4d46-a1f9-e827d944a1a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:40.955698 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8cbeb8-4169-4d46-a1f9-e827d944a1a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:43:40.958848 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8cbeb8-4169-4d46-a1f9-e827d944a1a0 service cinder] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:43:40.973367 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1398/2748] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:43:40 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:43:41.002881 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a916b6ec-7ab4-4fc8-ad74-881a6ed21af8 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:43:41.082628 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a916b6ec-7ab4-4fc8-ad74-881a6ed21af8 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:43:41.087584 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3865ebe9-5e7f-4950-ab14-33d61e6de083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:43:41.088601 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3865ebe9-5e7f-4950-ab14-33d61e6de083 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:41.089034 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3865ebe9-5e7f-4950-ab14-33d61e6de083 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:41.089339 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3865ebe9-5e7f-4950-ab14-33d61e6de083 admin admin] PATH_INFO: `/v3/users/51bbb75a20334a2ca4b87970aeb06945` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:41.090808 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3865ebe9-5e7f-4950-ab14-33d61e6de083 admin admin] RBAC: Authorizing `identity:delete_user(user_id=51bbb75a20334a2ca4b87970aeb06945)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:43:41.092898 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3865ebe9-5e7f-4950-ab14-33d61e6de083 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:43:41.357954 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-3865ebe9-5e7f-4950-ab14-33d61e6de083 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51bbb75a20334a2ca4b87970aeb06945'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:41.468700 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-3865ebe9-5e7f-4950-ab14-33d61e6de083 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51bbb75a20334a2ca4b87970aeb06945'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:41.469324 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-3865ebe9-5e7f-4950-ab14-33d61e6de083 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51bbb75a20334a2ca4b87970aeb06945'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:41.482548 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-3865ebe9-5e7f-4950-ab14-33d61e6de083 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51bbb75a20334a2ca4b87970aeb06945'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:41.483188 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-3865ebe9-5e7f-4950-ab14-33d61e6de083 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51bbb75a20334a2ca4b87970aeb06945'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:41.495941 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1351/2749] 10.222.0.66 () {58 vars in 1223 bytes} [Mon Jul 18 02:43:41 2022] DELETE /identity/v3/users/51bbb75a20334a2ca4b87970aeb06945 => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:43:41.501428 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b612446-af5e-4480-adaf-69e324dfc399 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:41.501924 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b612446-af5e-4480-adaf-69e324dfc399 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:41.502019 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b612446-af5e-4480-adaf-69e324dfc399 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:41.524315 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-4b612446-af5e-4480-adaf-69e324dfc399 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:43:41.524315 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-4b612446-af5e-4480-adaf-69e324dfc399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:43:41.585262 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1399/2750] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:43:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:43:41.595266 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3865ebe9-5e7f-4950-ab14-33d61e6de083 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:43:41.596698 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-3865ebe9-5e7f-4950-ab14-33d61e6de083 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:43:41.609473 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:43:41.661011 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3865ebe9-5e7f-4950-ab14-33d61e6de083 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:43:41.664803 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f158f7ea-06e6-448d-bbb8-38df40735e14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eaf19a259c2741cea375a38763e33cdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:43:41.665456 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f158f7ea-06e6-448d-bbb8-38df40735e14 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:41.665743 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f158f7ea-06e6-448d-bbb8-38df40735e14 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:41.666022 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f158f7ea-06e6-448d-bbb8-38df40735e14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:41.676515 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f158f7ea-06e6-448d-bbb8-38df40735e14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:43:41.678030 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f158f7ea-06e6-448d-bbb8-38df40735e14 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:43:41.690211 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1352/2751] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:43:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:43:42.194891 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b612446-af5e-4480-adaf-69e324dfc399 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:43:42.196419 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-4b612446-af5e-4480-adaf-69e324dfc399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:43:42.217128 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b612446-af5e-4480-adaf-69e324dfc399 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:43:42.223663 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b02c13-579a-4fb6-b436-44c31b917a27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:43:42.224541 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b02c13-579a-4fb6-b436-44c31b917a27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:42.224870 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b02c13-579a-4fb6-b436-44c31b917a27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:42.225146 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b02c13-579a-4fb6-b436-44c31b917a27 admin admin] PATH_INFO: `/v3/projects/49e6c9938c6145db9c4af1094aae8985` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:42.226353 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b02c13-579a-4fb6-b436-44c31b917a27 admin admin] RBAC: Authorizing `identity:delete_project(project_id=49e6c9938c6145db9c4af1094aae8985)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:43:42.228317 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b02c13-579a-4fb6-b436-44c31b917a27 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:43:42.384413 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-38b02c13-579a-4fb6-b436-44c31b917a27 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49e6c9938c6145db9c4af1094aae8985', 'request_id': 'req-38b02c13-579a-4fb6-b436-44c31b917a27'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:42.390707 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-38b02c13-579a-4fb6-b436-44c31b917a27 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49e6c9938c6145db9c4af1094aae8985', 'request_id': 'req-38b02c13-579a-4fb6-b436-44c31b917a27'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:42.520592 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-38b02c13-579a-4fb6-b436-44c31b917a27 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49e6c9938c6145db9c4af1094aae8985', 'request_id': 'req-38b02c13-579a-4fb6-b436-44c31b917a27'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:42.533417 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-38b02c13-579a-4fb6-b436-44c31b917a27 admin admin] The token cache is being invalidate because project 49e6c9938c6145db9c4af1094aae8985 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 02:43:42.533702 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-38b02c13-579a-4fb6-b436-44c31b917a27 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:42.534361 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-38b02c13-579a-4fb6-b436-44c31b917a27 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:42.535906 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1400/2752] 10.222.0.66 () {58 vars in 1232 bytes} [Mon Jul 18 02:43:42 2022] DELETE /identity/v3/projects/49e6c9938c6145db9c4af1094aae8985 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:43:42.540080 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f158f7ea-06e6-448d-bbb8-38df40735e14 service neutron] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:43:42.550028 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f158f7ea-06e6-448d-bbb8-38df40735e14 service neutron] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:43:42.616732 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad509e0-430d-453d-bcde-1b9817b966aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:43:42.617614 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad509e0-430d-453d-bcde-1b9817b966aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:42.617900 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad509e0-430d-453d-bcde-1b9817b966aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:42.618236 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad509e0-430d-453d-bcde-1b9817b966aa admin admin] PATH_INFO: `/v3/users/4e7cb443b0f743ce96a9f9bd866637d5` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:42.619462 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad509e0-430d-453d-bcde-1b9817b966aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e7cb443b0f743ce96a9f9bd866637d5)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:43:42.621394 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad509e0-430d-453d-bcde-1b9817b966aa admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:43:42.835693 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-bad509e0-430d-453d-bcde-1b9817b966aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e7cb443b0f743ce96a9f9bd866637d5'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:42.949161 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-bad509e0-430d-453d-bcde-1b9817b966aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e7cb443b0f743ce96a9f9bd866637d5'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:42.949796 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-bad509e0-430d-453d-bcde-1b9817b966aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e7cb443b0f743ce96a9f9bd866637d5'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:42.963536 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-bad509e0-430d-453d-bcde-1b9817b966aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e7cb443b0f743ce96a9f9bd866637d5'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:42.964311 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-bad509e0-430d-453d-bcde-1b9817b966aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e7cb443b0f743ce96a9f9bd866637d5'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:42.977602 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1353/2753] 10.222.0.66 () {58 vars in 1223 bytes} [Mon Jul 18 02:43:42 2022] DELETE /identity/v3/users/4e7cb443b0f743ce96a9f9bd866637d5 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:43:43.553009 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b02c13-579a-4fb6-b436-44c31b917a27 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:43:43.553009 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-38b02c13-579a-4fb6-b436-44c31b917a27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:43:43.573166 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b02c13-579a-4fb6-b436-44c31b917a27 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:43:43.626383 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:43:43.633203 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe005393-430a-48ec-a230-2431cf84756b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:43:43.634076 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe005393-430a-48ec-a230-2431cf84756b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:43.634389 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe005393-430a-48ec-a230-2431cf84756b admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:43.634669 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe005393-430a-48ec-a230-2431cf84756b admin admin] PATH_INFO: `/v3/projects/46406bc0cf33427c9714556c833de65d` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:43.635856 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe005393-430a-48ec-a230-2431cf84756b admin admin] RBAC: Authorizing `identity:delete_project(project_id=46406bc0cf33427c9714556c833de65d)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:43:43.637825 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe005393-430a-48ec-a230-2431cf84756b admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:43:43.795599 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-fe005393-430a-48ec-a230-2431cf84756b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46406bc0cf33427c9714556c833de65d', 'request_id': 'req-fe005393-430a-48ec-a230-2431cf84756b'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:43.801064 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-fe005393-430a-48ec-a230-2431cf84756b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46406bc0cf33427c9714556c833de65d', 'request_id': 'req-fe005393-430a-48ec-a230-2431cf84756b'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:43.930360 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-fe005393-430a-48ec-a230-2431cf84756b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46406bc0cf33427c9714556c833de65d', 'request_id': 'req-fe005393-430a-48ec-a230-2431cf84756b'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:43.943001 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-fe005393-430a-48ec-a230-2431cf84756b admin admin] The token cache is being invalidate because project 46406bc0cf33427c9714556c833de65d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 02:43:43.943288 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-fe005393-430a-48ec-a230-2431cf84756b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:43.943917 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-fe005393-430a-48ec-a230-2431cf84756b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:43.945474 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1401/2754] 10.222.0.66 () {58 vars in 1232 bytes} [Mon Jul 18 02:43:43 2022] DELETE /identity/v3/projects/46406bc0cf33427c9714556c833de65d => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:43:43.950438 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd13583-3781-41d4-ba41-e4b6d8c9c0f4 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:43.950950 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd13583-3781-41d4-ba41-e4b6d8c9c0f4 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:43.951062 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd13583-3781-41d4-ba41-e4b6d8c9c0f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:43.968873 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-2cd13583-3781-41d4-ba41-e4b6d8c9c0f4 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:43:43.970071 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-2cd13583-3781-41d4-ba41-e4b6d8c9c0f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:43:43.994734 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:43:44.030296 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1354/2755] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:43:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:43:44.039178 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd766bd8-9b5a-43a2-82e1-36fbbceb673b None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:44.039470 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd766bd8-9b5a-43a2-82e1-36fbbceb673b None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:44.039755 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd766bd8-9b5a-43a2-82e1-36fbbceb673b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:44.058911 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-bd766bd8-9b5a-43a2-82e1-36fbbceb673b None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:43:44.060797 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-bd766bd8-9b5a-43a2-82e1-36fbbceb673b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:43:44.069100 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1402/2756] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:43:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 02:43:44.073459 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd13583-3781-41d4-ba41-e4b6d8c9c0f4 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:43:44.084028 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd13583-3781-41d4-ba41-e4b6d8c9c0f4 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:43:44.088546 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14c4a32-337f-4e80-ad2b-fca2562158e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:43:44.089805 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14c4a32-337f-4e80-ad2b-fca2562158e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:44.089805 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14c4a32-337f-4e80-ad2b-fca2562158e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:44.089805 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14c4a32-337f-4e80-ad2b-fca2562158e5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:44.090534 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14c4a32-337f-4e80-ad2b-fca2562158e5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:43:44.090842 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14c4a32-337f-4e80-ad2b-fca2562158e5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:43:44.092906 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14c4a32-337f-4e80-ad2b-fca2562158e5 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:43:44.104471 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1355/2757] 10.222.0.66 () {58 vars in 1152 bytes} [Mon Jul 18 02:43:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:43:44.109438 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd766bd8-9b5a-43a2-82e1-36fbbceb673b None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:43:44.118944 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd766bd8-9b5a-43a2-82e1-36fbbceb673b None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:43:44.124620 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb53f7ed-b1af-43f1-a618-f1e90d313290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:43:44.125270 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb53f7ed-b1af-43f1-a618-f1e90d313290 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:44.125553 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb53f7ed-b1af-43f1-a618-f1e90d313290 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:44.125827 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb53f7ed-b1af-43f1-a618-f1e90d313290 admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:44.126624 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb53f7ed-b1af-43f1-a618-f1e90d313290 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:43:44.128515 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb53f7ed-b1af-43f1-a618-f1e90d313290 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:43:44.259061 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1403/2758] 10.222.0.66 () {60 vars in 1152 bytes} [Mon Jul 18 02:43:44 2022] POST /identity/v3/projects => generated 399 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:43:44.263226 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14c4a32-337f-4e80-ad2b-fca2562158e5 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:43:44.273572 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14c4a32-337f-4e80-ad2b-fca2562158e5 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:43:44.330843 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01336b7-7270-46f5-9820-82aaa42c5e57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:43:44.331515 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01336b7-7270-46f5-9820-82aaa42c5e57 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:44.331814 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01336b7-7270-46f5-9820-82aaa42c5e57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:44.332072 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01336b7-7270-46f5-9820-82aaa42c5e57 admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:44.332753 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01336b7-7270-46f5-9820-82aaa42c5e57 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:43:44.334701 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01336b7-7270-46f5-9820-82aaa42c5e57 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:43:44.446784 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1356/2759] 10.222.0.66 () {60 vars in 1143 bytes} [Mon Jul 18 02:43:44 2022] POST /identity/v3/users => generated 300 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:43:44.451382 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb53f7ed-b1af-43f1-a618-f1e90d313290 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:43:44.462244 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb53f7ed-b1af-43f1-a618-f1e90d313290 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:43:44.467477 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8b2dcd-a996-49e3-8b2e-1732610f7174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:43:44.468143 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8b2dcd-a996-49e3-8b2e-1732610f7174 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:44.468428 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8b2dcd-a996-49e3-8b2e-1732610f7174 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:44.468703 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8b2dcd-a996-49e3-8b2e-1732610f7174 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:44.469427 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8b2dcd-a996-49e3-8b2e-1732610f7174 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:43:44.471082 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8b2dcd-a996-49e3-8b2e-1732610f7174 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:43:44.483307 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1404/2760] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:43:44 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:43:44.487326 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01336b7-7270-46f5-9820-82aaa42c5e57 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:43:44.497563 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01336b7-7270-46f5-9820-82aaa42c5e57 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:43:44.502332 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d32a27e-8359-47fe-b184-b9b56c4fd455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:43:44.502993 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d32a27e-8359-47fe-b184-b9b56c4fd455 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:44.503278 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d32a27e-8359-47fe-b184-b9b56c4fd455 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:44.503560 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d32a27e-8359-47fe-b184-b9b56c4fd455 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:44.504284 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d32a27e-8359-47fe-b184-b9b56c4fd455 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:43:44.505893 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d32a27e-8359-47fe-b184-b9b56c4fd455 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:43:44.516154 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1357/2761] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:43:44 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:43:44.520112 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8b2dcd-a996-49e3-8b2e-1732610f7174 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:43:44.530668 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8b2dcd-a996-49e3-8b2e-1732610f7174 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:43:44.535393 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbbdabc-a0f3-4f9c-9193-707412ede5a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:43:44.536273 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbbdabc-a0f3-4f9c-9193-707412ede5a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:44.536557 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbbdabc-a0f3-4f9c-9193-707412ede5a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:44.536838 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbbdabc-a0f3-4f9c-9193-707412ede5a1 admin admin] PATH_INFO: `/v3/projects/83295ffc33494fa983acca297992009f/users/1bd4b4188f6c4203b07feed68720509a/roles/65feabf3b27944a8b01a86c8ac608457` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:44.552489 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbbdabc-a0f3-4f9c-9193-707412ede5a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=83295ffc33494fa983acca297992009f, user_id=1bd4b4188f6c4203b07feed68720509a, role_id=65feabf3b27944a8b01a86c8ac608457)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:43:44.555198 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbbdabc-a0f3-4f9c-9193-707412ede5a1 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:43:44.793945 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1405/2762] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:43:44 2022] PUT /identity/v3/projects/83295ffc33494fa983acca297992009f/users/1bd4b4188f6c4203b07feed68720509a/roles/65feabf3b27944a8b01a86c8ac608457 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:43:44.822931 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a4cec1-76b1-4d43-b77e-258ddffebef8 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:44.823159 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a4cec1-76b1-4d43-b77e-258ddffebef8 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:44.823443 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a4cec1-76b1-4d43-b77e-258ddffebef8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:44.850637 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-72a4cec1-76b1-4d43-b77e-258ddffebef8 None None] MFA Rules not processed for user `1bd4b4188f6c4203b07feed68720509a`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:43:44.854217 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-72a4cec1-76b1-4d43-b77e-258ddffebef8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:43:44.920536 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1358/2763] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:43:44 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:43:44.925248 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbbdabc-a0f3-4f9c-9193-707412ede5a1 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:43:44.988648 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbbdabc-a0f3-4f9c-9193-707412ede5a1 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:43:44.993183 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c82ba4-e6ef-4a12-89b5-f34a55770f7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:43:44.993842 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c82ba4-e6ef-4a12-89b5-f34a55770f7e admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:44.994154 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c82ba4-e6ef-4a12-89b5-f34a55770f7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:44.994437 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c82ba4-e6ef-4a12-89b5-f34a55770f7e admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:44.995128 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c82ba4-e6ef-4a12-89b5-f34a55770f7e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:43:44.997024 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c82ba4-e6ef-4a12-89b5-f34a55770f7e admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:43:45.068594 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a4cec1-76b1-4d43-b77e-258ddffebef8 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:43:45.069982 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-72a4cec1-76b1-4d43-b77e-258ddffebef8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:43:45.085069 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1406/2764] 10.222.0.66 () {60 vars in 1152 bytes} [Mon Jul 18 02:43:44 2022] POST /identity/v3/projects => generated 399 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:43:45.086205 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c82ba4-e6ef-4a12-89b5-f34a55770f7e admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:43:45.096757 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:43:45.099092 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:43:45.150325 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c82ba4-e6ef-4a12-89b5-f34a55770f7e admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:43:45.156434 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac1c35e-850c-4412-89ea-6979fc938c3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:43:45.157110 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac1c35e-850c-4412-89ea-6979fc938c3b admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:45.157498 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac1c35e-850c-4412-89ea-6979fc938c3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:45.157691 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac1c35e-850c-4412-89ea-6979fc938c3b admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:45.158566 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac1c35e-850c-4412-89ea-6979fc938c3b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:43:45.160614 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac1c35e-850c-4412-89ea-6979fc938c3b admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:43:45.162734 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a4cec1-76b1-4d43-b77e-258ddffebef8 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:43:45.223313 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46af16e8-e5be-4851-a5f6-4b8a8cd45c7d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88936b31709b429f8b6777c673ed7b8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:43:45.223966 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46af16e8-e5be-4851-a5f6-4b8a8cd45c7d service placement] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:45.224252 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46af16e8-e5be-4851-a5f6-4b8a8cd45c7d service placement] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:45.224531 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46af16e8-e5be-4851-a5f6-4b8a8cd45c7d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:45.226342 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-46af16e8-e5be-4851-a5f6-4b8a8cd45c7d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:43:45.238626 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46af16e8-e5be-4851-a5f6-4b8a8cd45c7d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:43:45.244106 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46af16e8-e5be-4851-a5f6-4b8a8cd45c7d service placement] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:43:45.250151 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1359/2765] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:43:45 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 02:43:45.281219 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1407/2766] 10.222.0.66 () {60 vars in 1143 bytes} [Mon Jul 18 02:43:45 2022] POST /identity/v3/users => generated 306 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:43:45.285336 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46af16e8-e5be-4851-a5f6-4b8a8cd45c7d service placement] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:43:45.296642 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46af16e8-e5be-4851-a5f6-4b8a8cd45c7d service placement] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:43:45.301447 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22eb966-db4e-4762-b134-6290cd7f8664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:43:45.302165 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22eb966-db4e-4762-b134-6290cd7f8664 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:45.302453 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22eb966-db4e-4762-b134-6290cd7f8664 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:45.302729 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22eb966-db4e-4762-b134-6290cd7f8664 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:45.303448 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22eb966-db4e-4762-b134-6290cd7f8664 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:43:45.305113 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22eb966-db4e-4762-b134-6290cd7f8664 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:43:45.316658 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1360/2767] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:43:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:43:45.320348 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac1c35e-850c-4412-89ea-6979fc938c3b admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:43:45.330625 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac1c35e-850c-4412-89ea-6979fc938c3b admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:43:45.341470 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3b72d7-525d-4921-bbc2-eb14635598bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:43:45.341470 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3b72d7-525d-4921-bbc2-eb14635598bd admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:45.341470 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3b72d7-525d-4921-bbc2-eb14635598bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:45.341470 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3b72d7-525d-4921-bbc2-eb14635598bd admin admin] PATH_INFO: `/v3/projects/2c4faa339b7a4eeba8ce8d7f4e8a0ae8/users/ca0c1f010e04483cb1d05971083b09bc/roles/d4aa3324f0c84887b41d0dc4acf7f30e` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:45.352882 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3b72d7-525d-4921-bbc2-eb14635598bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c4faa339b7a4eeba8ce8d7f4e8a0ae8, user_id=ca0c1f010e04483cb1d05971083b09bc, role_id=d4aa3324f0c84887b41d0dc4acf7f30e)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:43:45.356416 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3b72d7-525d-4921-bbc2-eb14635598bd admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:43:45.509745 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1408/2768] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:43:45 2022] PUT /identity/v3/projects/2c4faa339b7a4eeba8ce8d7f4e8a0ae8/users/ca0c1f010e04483cb1d05971083b09bc/roles/d4aa3324f0c84887b41d0dc4acf7f30e => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:43:45.514568 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22eb966-db4e-4762-b134-6290cd7f8664 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:43:45.524846 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22eb966-db4e-4762-b134-6290cd7f8664 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:43:45.581307 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1752c6b2-43ee-46c8-a12a-a7afe32ba9c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:43:45.581975 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1752c6b2-43ee-46c8-a12a-a7afe32ba9c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:45.582282 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1752c6b2-43ee-46c8-a12a-a7afe32ba9c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:45.582581 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1752c6b2-43ee-46c8-a12a-a7afe32ba9c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:45.583292 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1752c6b2-43ee-46c8-a12a-a7afe32ba9c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:43:45.584881 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1752c6b2-43ee-46c8-a12a-a7afe32ba9c5 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:43:45.595732 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1361/2769] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:43:45 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:43:45.599833 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3b72d7-525d-4921-bbc2-eb14635598bd admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:43:45.611479 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3b72d7-525d-4921-bbc2-eb14635598bd admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:43:45.621359 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125f66d3-b904-4f64-ae44-76ae37fa73a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:43:45.621359 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125f66d3-b904-4f64-ae44-76ae37fa73a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:45.621359 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125f66d3-b904-4f64-ae44-76ae37fa73a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:45.621359 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125f66d3-b904-4f64-ae44-76ae37fa73a6 admin admin] PATH_INFO: `/v3/projects/2c4faa339b7a4eeba8ce8d7f4e8a0ae8/users/ca0c1f010e04483cb1d05971083b09bc/roles/d4aa3324f0c84887b41d0dc4acf7f30e` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:45.622241 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125f66d3-b904-4f64-ae44-76ae37fa73a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c4faa339b7a4eeba8ce8d7f4e8a0ae8, user_id=ca0c1f010e04483cb1d05971083b09bc, role_id=d4aa3324f0c84887b41d0dc4acf7f30e)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:43:45.623688 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125f66d3-b904-4f64-ae44-76ae37fa73a6 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:43:45.631429 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1752c6b2-43ee-46c8-a12a-a7afe32ba9c5 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:43:45.633751 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-1752c6b2-43ee-46c8-a12a-a7afe32ba9c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:43:45.635745 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1409/2770] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:43:45 2022] PUT /identity/v3/projects/2c4faa339b7a4eeba8ce8d7f4e8a0ae8/users/ca0c1f010e04483cb1d05971083b09bc/roles/d4aa3324f0c84887b41d0dc4acf7f30e => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:43:45.647120 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb042521-a0dc-47b1-b620-c153075d6fca None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:45.647412 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb042521-a0dc-47b1-b620-c153075d6fca None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:45.647689 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb042521-a0dc-47b1-b620-c153075d6fca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:45.671829 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:43:45.675947 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-cb042521-a0dc-47b1-b620-c153075d6fca None None] MFA Rules not processed for user `ca0c1f010e04483cb1d05971083b09bc`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:43:45.677443 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-cb042521-a0dc-47b1-b620-c153075d6fca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:43:45.707590 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1752c6b2-43ee-46c8-a12a-a7afe32ba9c5 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:43:45.754814 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1410/2771] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:43:45 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 02:43:45.762698 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb042521-a0dc-47b1-b620-c153075d6fca None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:43:45.764720 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-cb042521-a0dc-47b1-b620-c153075d6fca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:43:45.768200 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ee7fb2-4d4f-4f25-91fa-5fee7c755b13 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd49388facdf446885e55b45ef91f01e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:43:45.768995 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ee7fb2-4d4f-4f25-91fa-5fee7c755b13 service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:45.769220 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ee7fb2-4d4f-4f25-91fa-5fee7c755b13 service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:45.769412 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ee7fb2-4d4f-4f25-91fa-5fee7c755b13 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:45.771320 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-f0ee7fb2-4d4f-4f25-91fa-5fee7c755b13 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:43:45.777901 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb042521-a0dc-47b1-b620-c153075d6fca None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:43:45.781681 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb0ae3b-163f-433a-8498-7884c38f0b30 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd49388facdf446885e55b45ef91f01e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:43:45.782352 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb0ae3b-163f-433a-8498-7884c38f0b30 service cinder] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:45.782634 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb0ae3b-163f-433a-8498-7884c38f0b30 service cinder] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:45.782912 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb0ae3b-163f-433a-8498-7884c38f0b30 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:45.793236 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb0ae3b-163f-433a-8498-7884c38f0b30 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:43:45.794766 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb0ae3b-163f-433a-8498-7884c38f0b30 service cinder] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:43:45.804608 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1411/2772] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:43:45 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 02:43:45.833552 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ee7fb2-4d4f-4f25-91fa-5fee7c755b13 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:43:45.835021 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ee7fb2-4d4f-4f25-91fa-5fee7c755b13 service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:43:45.844024 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1362/2773] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:43:45 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:43:46.121874 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb0ae3b-163f-433a-8498-7884c38f0b30 service cinder] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:43:46.131335 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb0ae3b-163f-433a-8498-7884c38f0b30 service cinder] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:43:46.135183 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ac873c-3df8-45b7-9788-36b56b3097f9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd49388facdf446885e55b45ef91f01e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:43:46.135816 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ac873c-3df8-45b7-9788-36b56b3097f9 service cinder] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:46.136101 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ac873c-3df8-45b7-9788-36b56b3097f9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:46.136406 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ac873c-3df8-45b7-9788-36b56b3097f9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:46.166497 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:43:46.194496 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ac873c-3df8-45b7-9788-36b56b3097f9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:43:46.195959 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ac873c-3df8-45b7-9788-36b56b3097f9 service cinder] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:43:46.204793 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1412/2774] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:43:46 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:43:54.719231 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0531a6e-6a92-4e59-9834-149d4bfd5e21 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:54.719770 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0531a6e-6a92-4e59-9834-149d4bfd5e21 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:54.719770 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0531a6e-6a92-4e59-9834-149d4bfd5e21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:54.737781 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-b0531a6e-6a92-4e59-9834-149d4bfd5e21 None None] MFA Rules not processed for user `41a452eca3af46b09f56a80fa8652247`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:43:54.739116 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-b0531a6e-6a92-4e59-9834-149d4bfd5e21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:43:54.746322 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1363/2775] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:43:54 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:43:54.988077 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ac873c-3df8-45b7-9788-36b56b3097f9 service cinder] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:43:55.052801 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ac873c-3df8-45b7-9788-36b56b3097f9 service cinder] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:43:55.057262 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0eb046-f854-4fab-a2e0-f38ba26f5790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:43:55.058148 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0eb046-f854-4fab-a2e0-f38ba26f5790 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:55.058429 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0eb046-f854-4fab-a2e0-f38ba26f5790 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:55.058703 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0eb046-f854-4fab-a2e0-f38ba26f5790 admin admin] PATH_INFO: `/v3/users/1bd4b4188f6c4203b07feed68720509a` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:55.059914 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0eb046-f854-4fab-a2e0-f38ba26f5790 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1bd4b4188f6c4203b07feed68720509a)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:43:55.062157 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0eb046-f854-4fab-a2e0-f38ba26f5790 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:43:55.315875 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-fc0eb046-f854-4fab-a2e0-f38ba26f5790 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1bd4b4188f6c4203b07feed68720509a'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:55.409927 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-fc0eb046-f854-4fab-a2e0-f38ba26f5790 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1bd4b4188f6c4203b07feed68720509a'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:55.417469 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-fc0eb046-f854-4fab-a2e0-f38ba26f5790 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1bd4b4188f6c4203b07feed68720509a'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:55.418121 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-fc0eb046-f854-4fab-a2e0-f38ba26f5790 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1bd4b4188f6c4203b07feed68720509a'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:55.429994 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-fc0eb046-f854-4fab-a2e0-f38ba26f5790 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1bd4b4188f6c4203b07feed68720509a'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:55.436078 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1413/2776] 10.222.0.66 () {58 vars in 1223 bytes} [Mon Jul 18 02:43:54 2022] DELETE /identity/v3/users/1bd4b4188f6c4203b07feed68720509a => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:43:55.440818 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351c5729-120e-455e-8a9a-38a1406484c0 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:55.441128 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351c5729-120e-455e-8a9a-38a1406484c0 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:55.441405 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351c5729-120e-455e-8a9a-38a1406484c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:55.459401 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-351c5729-120e-455e-8a9a-38a1406484c0 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:43:55.460588 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-351c5729-120e-455e-8a9a-38a1406484c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:43:55.502255 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0eb046-f854-4fab-a2e0-f38ba26f5790 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:43:55.503858 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-fc0eb046-f854-4fab-a2e0-f38ba26f5790 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:43:55.522974 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1364/2777] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:43:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:43:55.532797 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351c5729-120e-455e-8a9a-38a1406484c0 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:43:55.534653 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-351c5729-120e-455e-8a9a-38a1406484c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:43:55.557438 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:43:55.566750 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:43:55.572947 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0eb046-f854-4fab-a2e0-f38ba26f5790 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:43:55.578280 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fdc2a6-255a-41f2-859b-f2416e3a34a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:43:55.579155 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fdc2a6-255a-41f2-859b-f2416e3a34a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:55.579440 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fdc2a6-255a-41f2-859b-f2416e3a34a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:55.579719 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fdc2a6-255a-41f2-859b-f2416e3a34a9 admin admin] PATH_INFO: `/v3/users/201a447a65a54dcb8bf85cc1cbb17d9c` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:55.580925 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fdc2a6-255a-41f2-859b-f2416e3a34a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=201a447a65a54dcb8bf85cc1cbb17d9c)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:43:55.582882 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fdc2a6-255a-41f2-859b-f2416e3a34a9 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:43:55.595841 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351c5729-120e-455e-8a9a-38a1406484c0 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:43:55.599831 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce102ec2-093d-4f72-b33f-a79943c56383 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eaf19a259c2741cea375a38763e33cdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:43:55.600492 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce102ec2-093d-4f72-b33f-a79943c56383 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:55.600776 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce102ec2-093d-4f72-b33f-a79943c56383 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:55.601053 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce102ec2-093d-4f72-b33f-a79943c56383 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:55.611467 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce102ec2-093d-4f72-b33f-a79943c56383 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:43:55.612961 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce102ec2-093d-4f72-b33f-a79943c56383 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:43:55.623053 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1365/2778] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:43:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 02:43:55.768663 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-39fdc2a6-255a-41f2-859b-f2416e3a34a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '201a447a65a54dcb8bf85cc1cbb17d9c'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:55.869390 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-39fdc2a6-255a-41f2-859b-f2416e3a34a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '201a447a65a54dcb8bf85cc1cbb17d9c'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:55.884690 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-39fdc2a6-255a-41f2-859b-f2416e3a34a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '201a447a65a54dcb8bf85cc1cbb17d9c'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:55.885530 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-39fdc2a6-255a-41f2-859b-f2416e3a34a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '201a447a65a54dcb8bf85cc1cbb17d9c'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:55.907526 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-39fdc2a6-255a-41f2-859b-f2416e3a34a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '201a447a65a54dcb8bf85cc1cbb17d9c'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:55.923705 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1414/2779] 10.222.0.66 () {58 vars in 1223 bytes} [Mon Jul 18 02:43:55 2022] DELETE /identity/v3/users/201a447a65a54dcb8bf85cc1cbb17d9c => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:43:55.928793 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bd9744-1781-42e9-8c8c-7d6468623346 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:55.929304 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bd9744-1781-42e9-8c8c-7d6468623346 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:55.929625 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bd9744-1781-42e9-8c8c-7d6468623346 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:55.949693 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-80bd9744-1781-42e9-8c8c-7d6468623346 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:43:55.951034 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-80bd9744-1781-42e9-8c8c-7d6468623346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:43:56.032623 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1366/2780] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:43:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:43:56.042200 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fdc2a6-255a-41f2-859b-f2416e3a34a9 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:43:56.043980 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-39fdc2a6-255a-41f2-859b-f2416e3a34a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:43:56.104230 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fdc2a6-255a-41f2-859b-f2416e3a34a9 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:43:56.108402 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e371a4-d81b-4718-9b64-7f39dc202937 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eaf19a259c2741cea375a38763e33cdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:43:56.109053 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e371a4-d81b-4718-9b64-7f39dc202937 service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:56.109338 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e371a4-d81b-4718-9b64-7f39dc202937 service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:56.109615 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e371a4-d81b-4718-9b64-7f39dc202937 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:56.119029 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e371a4-d81b-4718-9b64-7f39dc202937 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:43:56.120860 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e371a4-d81b-4718-9b64-7f39dc202937 service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:43:56.131037 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1415/2781] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:43:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:43:56.141814 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bd9744-1781-42e9-8c8c-7d6468623346 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:43:56.143249 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-80bd9744-1781-42e9-8c8c-7d6468623346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:43:56.155803 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bd9744-1781-42e9-8c8c-7d6468623346 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:43:56.161609 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42cf69e-36bd-425f-9b04-3577589e5485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:43:56.162743 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42cf69e-36bd-425f-9b04-3577589e5485 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:56.163035 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42cf69e-36bd-425f-9b04-3577589e5485 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:56.163301 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42cf69e-36bd-425f-9b04-3577589e5485 admin admin] PATH_INFO: `/v3/projects/83295ffc33494fa983acca297992009f` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:56.164503 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42cf69e-36bd-425f-9b04-3577589e5485 admin admin] RBAC: Authorizing `identity:delete_project(project_id=83295ffc33494fa983acca297992009f)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:43:56.167377 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42cf69e-36bd-425f-9b04-3577589e5485 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:43:56.297738 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-b42cf69e-36bd-425f-9b04-3577589e5485 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83295ffc33494fa983acca297992009f', 'request_id': 'req-b42cf69e-36bd-425f-9b04-3577589e5485'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:56.303448 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-b42cf69e-36bd-425f-9b04-3577589e5485 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83295ffc33494fa983acca297992009f', 'request_id': 'req-b42cf69e-36bd-425f-9b04-3577589e5485'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:56.309626 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-b42cf69e-36bd-425f-9b04-3577589e5485 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83295ffc33494fa983acca297992009f', 'request_id': 'req-b42cf69e-36bd-425f-9b04-3577589e5485'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:56.414217 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-b42cf69e-36bd-425f-9b04-3577589e5485 admin admin] The token cache is being invalidate because project 83295ffc33494fa983acca297992009f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 02:43:56.415578 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-b42cf69e-36bd-425f-9b04-3577589e5485 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:56.415926 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-b42cf69e-36bd-425f-9b04-3577589e5485 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:56.421347 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1367/2782] 10.222.0.66 () {58 vars in 1232 bytes} [Mon Jul 18 02:43:56 2022] DELETE /identity/v3/projects/83295ffc33494fa983acca297992009f => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:43:56.423207 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e371a4-d81b-4718-9b64-7f39dc202937 service neutron] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:43:56.434564 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e371a4-d81b-4718-9b64-7f39dc202937 service neutron] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:43:56.623035 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101b6a2b-6355-4885-9c80-1fe595686c5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:43:56.624546 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101b6a2b-6355-4885-9c80-1fe595686c5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:56.624826 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101b6a2b-6355-4885-9c80-1fe595686c5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:56.625944 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101b6a2b-6355-4885-9c80-1fe595686c5b admin admin] PATH_INFO: `/v3/users/ca0c1f010e04483cb1d05971083b09bc` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:56.627233 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101b6a2b-6355-4885-9c80-1fe595686c5b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca0c1f010e04483cb1d05971083b09bc)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:43:56.629134 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101b6a2b-6355-4885-9c80-1fe595686c5b admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:43:56.641457 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:43:56.671807 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42cf69e-36bd-425f-9b04-3577589e5485 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:43:56.673276 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-b42cf69e-36bd-425f-9b04-3577589e5485 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:43:56.687939 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42cf69e-36bd-425f-9b04-3577589e5485 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:43:56.692896 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d9ff1e-6522-4b01-8d8b-ccb65971613d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:43:56.693761 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d9ff1e-6522-4b01-8d8b-ccb65971613d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:56.694038 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d9ff1e-6522-4b01-8d8b-ccb65971613d admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:56.694383 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d9ff1e-6522-4b01-8d8b-ccb65971613d admin admin] PATH_INFO: `/v3/projects/7fc721738d474d0fa344aa5e80f66869` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:56.695940 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d9ff1e-6522-4b01-8d8b-ccb65971613d admin admin] RBAC: Authorizing `identity:delete_project(project_id=7fc721738d474d0fa344aa5e80f66869)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:43:56.701873 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d9ff1e-6522-4b01-8d8b-ccb65971613d admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:43:56.875543 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:43:56.876712 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-c1d9ff1e-6522-4b01-8d8b-ccb65971613d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7fc721738d474d0fa344aa5e80f66869', 'request_id': 'req-c1d9ff1e-6522-4b01-8d8b-ccb65971613d'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:56.882659 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-c1d9ff1e-6522-4b01-8d8b-ccb65971613d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7fc721738d474d0fa344aa5e80f66869', 'request_id': 'req-c1d9ff1e-6522-4b01-8d8b-ccb65971613d'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:56.889486 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-c1d9ff1e-6522-4b01-8d8b-ccb65971613d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7fc721738d474d0fa344aa5e80f66869', 'request_id': 'req-c1d9ff1e-6522-4b01-8d8b-ccb65971613d'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:56.978531 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-c1d9ff1e-6522-4b01-8d8b-ccb65971613d admin admin] The token cache is being invalidate because project 7fc721738d474d0fa344aa5e80f66869 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 02:43:56.979189 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-c1d9ff1e-6522-4b01-8d8b-ccb65971613d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:56.980002 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-c1d9ff1e-6522-4b01-8d8b-ccb65971613d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:56.982321 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1368/2783] 10.222.0.66 () {58 vars in 1232 bytes} [Mon Jul 18 02:43:56 2022] DELETE /identity/v3/projects/7fc721738d474d0fa344aa5e80f66869 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 02:43:56.987675 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f391a190-3a62-427e-9c19-4788a21896bd None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:56.987965 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f391a190-3a62-427e-9c19-4788a21896bd None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:56.988280 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f391a190-3a62-427e-9c19-4788a21896bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:56.992196 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-101b6a2b-6355-4885-9c80-1fe595686c5b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca0c1f010e04483cb1d05971083b09bc'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:57.008608 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-f391a190-3a62-427e-9c19-4788a21896bd None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:43:57.009948 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-f391a190-3a62-427e-9c19-4788a21896bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:43:57.069486 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1369/2784] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:43:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 02:43:57.077588 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46bfbee-6520-4235-b69d-4c7b611cc289 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:57.077981 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46bfbee-6520-4235-b69d-4c7b611cc289 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:57.078190 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46bfbee-6520-4235-b69d-4c7b611cc289 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:57.092510 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-101b6a2b-6355-4885-9c80-1fe595686c5b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca0c1f010e04483cb1d05971083b09bc'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:57.097205 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-f46bfbee-6520-4235-b69d-4c7b611cc289 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:43:57.098648 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-f46bfbee-6520-4235-b69d-4c7b611cc289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:43:57.102377 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-101b6a2b-6355-4885-9c80-1fe595686c5b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca0c1f010e04483cb1d05971083b09bc'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:57.104679 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-101b6a2b-6355-4885-9c80-1fe595686c5b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca0c1f010e04483cb1d05971083b09bc'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:57.107187 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1370/2785] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:43:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 02:43:57.112098 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46bfbee-6520-4235-b69d-4c7b611cc289 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:43:57.117888 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-101b6a2b-6355-4885-9c80-1fe595686c5b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca0c1f010e04483cb1d05971083b09bc'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:57.121299 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46bfbee-6520-4235-b69d-4c7b611cc289 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:43:57.124918 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1416/2786] 10.222.0.66 () {58 vars in 1223 bytes} [Mon Jul 18 02:43:56 2022] DELETE /identity/v3/users/ca0c1f010e04483cb1d05971083b09bc => generated 0 bytes in 702 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:43:57.127141 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e858720c-d965-492b-9778-b8700cf5834b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:43:57.127753 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e858720c-d965-492b-9778-b8700cf5834b admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:57.128045 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e858720c-d965-492b-9778-b8700cf5834b admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:57.128319 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e858720c-d965-492b-9778-b8700cf5834b admin admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:57.129054 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e858720c-d965-492b-9778-b8700cf5834b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:43:57.129360 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e858720c-d965-492b-9778-b8700cf5834b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:43:57.131138 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e858720c-d965-492b-9778-b8700cf5834b admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:43:57.147777 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1371/2787] 10.222.0.66 () {58 vars in 1152 bytes} [Mon Jul 18 02:43:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:43:57.149160 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101b6a2b-6355-4885-9c80-1fe595686c5b admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:43:57.150710 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-101b6a2b-6355-4885-9c80-1fe595686c5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:43:57.223407 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101b6a2b-6355-4885-9c80-1fe595686c5b admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:43:57.228363 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87b5bca-8cf4-4650-94a3-95fd5e4ab2d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:43:57.229026 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87b5bca-8cf4-4650-94a3-95fd5e4ab2d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:57.229307 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87b5bca-8cf4-4650-94a3-95fd5e4ab2d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:57.229714 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87b5bca-8cf4-4650-94a3-95fd5e4ab2d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:57.231027 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87b5bca-8cf4-4650-94a3-95fd5e4ab2d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:43:57.232785 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87b5bca-8cf4-4650-94a3-95fd5e4ab2d6 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:43:57.334054 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1417/2788] 10.222.0.66 () {60 vars in 1152 bytes} [Mon Jul 18 02:43:57 2022] POST /identity/v3/projects => generated 383 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:43:57.338545 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e858720c-d965-492b-9778-b8700cf5834b admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:43:57.348830 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e858720c-d965-492b-9778-b8700cf5834b admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:43:57.414450 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f71d1c7-b3ea-4924-b691-57d998ae34eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:43:57.415131 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f71d1c7-b3ea-4924-b691-57d998ae34eb admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:57.415415 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f71d1c7-b3ea-4924-b691-57d998ae34eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:57.415827 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f71d1c7-b3ea-4924-b691-57d998ae34eb admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:57.416661 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f71d1c7-b3ea-4924-b691-57d998ae34eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:43:57.418744 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f71d1c7-b3ea-4924-b691-57d998ae34eb admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:43:57.509677 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1372/2789] 10.222.0.66 () {60 vars in 1143 bytes} [Mon Jul 18 02:43:57 2022] POST /identity/v3/users => generated 292 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:43:57.513627 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87b5bca-8cf4-4650-94a3-95fd5e4ab2d6 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:43:57.524153 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87b5bca-8cf4-4650-94a3-95fd5e4ab2d6 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:43:57.528928 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adabf33f-2c5b-424f-8e41-b966d229aa0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:43:57.529637 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adabf33f-2c5b-424f-8e41-b966d229aa0a admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:57.529971 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adabf33f-2c5b-424f-8e41-b966d229aa0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:57.530277 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adabf33f-2c5b-424f-8e41-b966d229aa0a admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:57.530995 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adabf33f-2c5b-424f-8e41-b966d229aa0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:43:57.532568 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adabf33f-2c5b-424f-8e41-b966d229aa0a admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:43:57.542533 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1418/2790] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:43:57 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:43:57.547331 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f71d1c7-b3ea-4924-b691-57d998ae34eb admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:43:57.557501 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f71d1c7-b3ea-4924-b691-57d998ae34eb admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:43:57.563667 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23d52be-6495-4f2b-bcc6-da1f8cf097ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:43:57.564619 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23d52be-6495-4f2b-bcc6-da1f8cf097ee admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:57.564906 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23d52be-6495-4f2b-bcc6-da1f8cf097ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:57.565185 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23d52be-6495-4f2b-bcc6-da1f8cf097ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:57.565929 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23d52be-6495-4f2b-bcc6-da1f8cf097ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:43:57.567565 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23d52be-6495-4f2b-bcc6-da1f8cf097ee admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:43:57.577442 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adabf33f-2c5b-424f-8e41-b966d229aa0a admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:43:57.579445 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-adabf33f-2c5b-424f-8e41-b966d229aa0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:43:57.581038 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1373/2791] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:43:57 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:43:57.593279 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23d52be-6495-4f2b-bcc6-da1f8cf097ee admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:43:57.594751 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adabf33f-2c5b-424f-8e41-b966d229aa0a admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:43:57.606460 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0727a16-eaf5-487f-8054-cb81a98d8835 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:43:57.607333 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0727a16-eaf5-487f-8054-cb81a98d8835 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:57.608173 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0727a16-eaf5-487f-8054-cb81a98d8835 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:57.608507 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0727a16-eaf5-487f-8054-cb81a98d8835 admin admin] PATH_INFO: `/v3/projects/2c4faa339b7a4eeba8ce8d7f4e8a0ae8` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:57.609957 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0727a16-eaf5-487f-8054-cb81a98d8835 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c4faa339b7a4eeba8ce8d7f4e8a0ae8)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:43:57.612516 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0727a16-eaf5-487f-8054-cb81a98d8835 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:43:57.613872 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23d52be-6495-4f2b-bcc6-da1f8cf097ee admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:43:57.619125 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be25d47-b40d-4c33-a13b-36e1ce17ab90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:43:57.620094 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be25d47-b40d-4c33-a13b-36e1ce17ab90 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:57.620454 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be25d47-b40d-4c33-a13b-36e1ce17ab90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:57.620807 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be25d47-b40d-4c33-a13b-36e1ce17ab90 admin admin] PATH_INFO: `/v3/projects/ad2b132c879f4f5e90593713a3f472c1/users/b065087a79524114b5cacb8c5fd3254f/roles/65feabf3b27944a8b01a86c8ac608457` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:57.638726 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be25d47-b40d-4c33-a13b-36e1ce17ab90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad2b132c879f4f5e90593713a3f472c1, user_id=b065087a79524114b5cacb8c5fd3254f, role_id=65feabf3b27944a8b01a86c8ac608457)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:43:57.641703 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be25d47-b40d-4c33-a13b-36e1ce17ab90 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:43:57.766678 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1374/2792] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:43:57 2022] PUT /identity/v3/projects/ad2b132c879f4f5e90593713a3f472c1/users/b065087a79524114b5cacb8c5fd3254f/roles/65feabf3b27944a8b01a86c8ac608457 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 02:43:57.771526 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4624035-bdd0-4d8a-9bdd-e88fe86c7fd6 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:57.771892 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4624035-bdd0-4d8a-9bdd-e88fe86c7fd6 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:57.772179 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4624035-bdd0-4d8a-9bdd-e88fe86c7fd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:57.774630 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-a0727a16-eaf5-487f-8054-cb81a98d8835 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c4faa339b7a4eeba8ce8d7f4e8a0ae8', 'request_id': 'req-a0727a16-eaf5-487f-8054-cb81a98d8835'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:57.782026 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-a0727a16-eaf5-487f-8054-cb81a98d8835 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c4faa339b7a4eeba8ce8d7f4e8a0ae8', 'request_id': 'req-a0727a16-eaf5-487f-8054-cb81a98d8835'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:57.784270 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:43:57.789923 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-f4624035-bdd0-4d8a-9bdd-e88fe86c7fd6 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:43:57.791152 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-f4624035-bdd0-4d8a-9bdd-e88fe86c7fd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:43:57.797088 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=92,keystone:DELETE=3,keystone:INSERT=5 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:43:57.850592 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1375/2793] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:43:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 02:43:57.860869 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4624035-bdd0-4d8a-9bdd-e88fe86c7fd6 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:43:57.862328 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-f4624035-bdd0-4d8a-9bdd-e88fe86c7fd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:43:57.886233 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-a0727a16-eaf5-487f-8054-cb81a98d8835 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c4faa339b7a4eeba8ce8d7f4e8a0ae8', 'request_id': 'req-a0727a16-eaf5-487f-8054-cb81a98d8835'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:57.898465 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-a0727a16-eaf5-487f-8054-cb81a98d8835 admin admin] The token cache is being invalidate because project 2c4faa339b7a4eeba8ce8d7f4e8a0ae8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 02:43:57.898685 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-a0727a16-eaf5-487f-8054-cb81a98d8835 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:57.899379 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-a0727a16-eaf5-487f-8054-cb81a98d8835 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:43:57.909090 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1419/2794] 10.222.0.66 () {58 vars in 1232 bytes} [Mon Jul 18 02:43:57 2022] DELETE /identity/v3/projects/2c4faa339b7a4eeba8ce8d7f4e8a0ae8 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 02:43:57.913762 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecc1ced-1ac6-4a7a-883c-754e7d9ed5f3 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:57.914057 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecc1ced-1ac6-4a7a-883c-754e7d9ed5f3 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:57.914372 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecc1ced-1ac6-4a7a-883c-754e7d9ed5f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:57.921927 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4624035-bdd0-4d8a-9bdd-e88fe86c7fd6 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:43:57.926229 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc60ad6-3ffc-4087-9285-163e1481e3e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eaf19a259c2741cea375a38763e33cdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:43:57.926947 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc60ad6-3ffc-4087-9285-163e1481e3e3 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:57.927236 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc60ad6-3ffc-4087-9285-163e1481e3e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:57.927580 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc60ad6-3ffc-4087-9285-163e1481e3e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:57.929404 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-5fc60ad6-3ffc-4087-9285-163e1481e3e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:43:57.932939 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-eecc1ced-1ac6-4a7a-883c-754e7d9ed5f3 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:43:57.934177 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-eecc1ced-1ac6-4a7a-883c-754e7d9ed5f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:43:57.942310 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc60ad6-3ffc-4087-9285-163e1481e3e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:43:57.944836 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc60ad6-3ffc-4087-9285-163e1481e3e3 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:43:57.947446 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1420/2795] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:43:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 02:43:57.950468 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb330e68-7084-47f5-8be8-3b72899ee4ff None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:57.950760 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb330e68-7084-47f5-8be8-3b72899ee4ff None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:57.951039 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb330e68-7084-47f5-8be8-3b72899ee4ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:57.954005 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1376/2796] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:43:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 02:43:57.973611 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-eb330e68-7084-47f5-8be8-3b72899ee4ff None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:43:57.974868 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-eb330e68-7084-47f5-8be8-3b72899ee4ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:43:57.984094 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1421/2797] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:43:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 02:43:57.988077 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc60ad6-3ffc-4087-9285-163e1481e3e3 service neutron] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:43:57.997987 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc60ad6-3ffc-4087-9285-163e1481e3e3 service neutron] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:43:58.003223 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6751460a-6f21-4cf1-84c3-ec926851912b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:43:58.003886 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6751460a-6f21-4cf1-84c3-ec926851912b admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:58.004173 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6751460a-6f21-4cf1-84c3-ec926851912b admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:58.004457 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6751460a-6f21-4cf1-84c3-ec926851912b admin admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:58.005237 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6751460a-6f21-4cf1-84c3-ec926851912b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:43:58.005544 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6751460a-6f21-4cf1-84c3-ec926851912b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:43:58.007186 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6751460a-6f21-4cf1-84c3-ec926851912b admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:43:58.020321 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1377/2798] 10.222.0.66 () {58 vars in 1152 bytes} [Mon Jul 18 02:43:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:43:58.030481 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb330e68-7084-47f5-8be8-3b72899ee4ff None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:43:58.040197 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb330e68-7084-47f5-8be8-3b72899ee4ff None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:43:58.045131 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdbb466-421f-4ede-b26f-e113f294eda1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:43:58.046529 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdbb466-421f-4ede-b26f-e113f294eda1 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:58.046820 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdbb466-421f-4ede-b26f-e113f294eda1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:58.047100 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdbb466-421f-4ede-b26f-e113f294eda1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:58.047806 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdbb466-421f-4ede-b26f-e113f294eda1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:43:58.049736 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdbb466-421f-4ede-b26f-e113f294eda1 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:43:58.178958 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1422/2799] 10.222.0.66 () {60 vars in 1152 bytes} [Mon Jul 18 02:43:58 2022] POST /identity/v3/projects => generated 387 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:43:58.183060 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6751460a-6f21-4cf1-84c3-ec926851912b admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:43:58.193430 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6751460a-6f21-4cf1-84c3-ec926851912b admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:43:58.250973 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c695abc0-dee1-4870-83f6-bf2e262e8b7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:43:58.251643 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c695abc0-dee1-4870-83f6-bf2e262e8b7e admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:58.251929 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c695abc0-dee1-4870-83f6-bf2e262e8b7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:58.252207 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c695abc0-dee1-4870-83f6-bf2e262e8b7e admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:58.252897 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c695abc0-dee1-4870-83f6-bf2e262e8b7e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:43:58.255024 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c695abc0-dee1-4870-83f6-bf2e262e8b7e admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:43:58.381188 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1378/2800] 10.222.0.66 () {60 vars in 1143 bytes} [Mon Jul 18 02:43:58 2022] POST /identity/v3/users => generated 294 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:43:58.382607 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdbb466-421f-4ede-b26f-e113f294eda1 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:43:58.393779 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdbb466-421f-4ede-b26f-e113f294eda1 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:43:58.398677 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827069c2-f03d-4e76-8cf4-8c69135047fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:43:58.399389 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827069c2-f03d-4e76-8cf4-8c69135047fe admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:58.399675 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827069c2-f03d-4e76-8cf4-8c69135047fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:58.399945 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827069c2-f03d-4e76-8cf4-8c69135047fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:58.400674 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827069c2-f03d-4e76-8cf4-8c69135047fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:43:58.402294 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827069c2-f03d-4e76-8cf4-8c69135047fe admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:43:58.414216 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1423/2801] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:43:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:43:58.419863 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c695abc0-dee1-4870-83f6-bf2e262e8b7e admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:43:58.430138 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c695abc0-dee1-4870-83f6-bf2e262e8b7e admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:43:58.435118 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62201644-732d-4af7-b8d7-0ce46025b96d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:43:58.435859 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62201644-732d-4af7-b8d7-0ce46025b96d admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:58.436351 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62201644-732d-4af7-b8d7-0ce46025b96d admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:58.436641 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62201644-732d-4af7-b8d7-0ce46025b96d admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:58.437377 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62201644-732d-4af7-b8d7-0ce46025b96d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:43:58.439045 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62201644-732d-4af7-b8d7-0ce46025b96d admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:43:58.452319 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1379/2802] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:43:58 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:43:58.456142 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827069c2-f03d-4e76-8cf4-8c69135047fe admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:43:58.468318 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827069c2-f03d-4e76-8cf4-8c69135047fe admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:43:58.473231 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf68e5c3-e509-4ce7-b299-3c14b109c8d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:43:58.474367 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf68e5c3-e509-4ce7-b299-3c14b109c8d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:58.475353 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf68e5c3-e509-4ce7-b299-3c14b109c8d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:58.475675 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf68e5c3-e509-4ce7-b299-3c14b109c8d6 admin admin] PATH_INFO: `/v3/projects/9a43c7e0e2934215b5000d957a1ddc40/users/e32ea64f74354ef3a5f14937b1a053bf/roles/65feabf3b27944a8b01a86c8ac608457` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:58.493755 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf68e5c3-e509-4ce7-b299-3c14b109c8d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a43c7e0e2934215b5000d957a1ddc40, user_id=e32ea64f74354ef3a5f14937b1a053bf, role_id=65feabf3b27944a8b01a86c8ac608457)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:43:58.495988 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf68e5c3-e509-4ce7-b299-3c14b109c8d6 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:43:58.583093 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1424/2803] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:43:58 2022] PUT /identity/v3/projects/9a43c7e0e2934215b5000d957a1ddc40/users/e32ea64f74354ef3a5f14937b1a053bf/roles/65feabf3b27944a8b01a86c8ac608457 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 02:43:58.593691 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2df4374-0e20-4a65-b305-f3f50e82e839 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:58.594128 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2df4374-0e20-4a65-b305-f3f50e82e839 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:58.594880 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2df4374-0e20-4a65-b305-f3f50e82e839 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:58.633548 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-d2df4374-0e20-4a65-b305-f3f50e82e839 None None] MFA Rules not processed for user `e32ea64f74354ef3a5f14937b1a053bf`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:43:58.635276 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-d2df4374-0e20-4a65-b305-f3f50e82e839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:43:58.646264 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:43:58.713544 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1380/2804] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:43:58 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:43:58.717995 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf68e5c3-e509-4ce7-b299-3c14b109c8d6 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:43:58.796092 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf68e5c3-e509-4ce7-b299-3c14b109c8d6 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:43:58.801046 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d58b47-03d0-4f30-9495-09edc177bf9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:43:58.801770 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d58b47-03d0-4f30-9495-09edc177bf9a admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:58.804838 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d58b47-03d0-4f30-9495-09edc177bf9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:58.804838 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d58b47-03d0-4f30-9495-09edc177bf9a admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:58.804838 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d58b47-03d0-4f30-9495-09edc177bf9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:43:58.805590 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d58b47-03d0-4f30-9495-09edc177bf9a admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:43:58.917068 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1425/2805] 10.222.0.66 () {60 vars in 1152 bytes} [Mon Jul 18 02:43:58 2022] POST /identity/v3/projects => generated 389 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:43:58.920466 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2df4374-0e20-4a65-b305-f3f50e82e839 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:43:58.932784 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2df4374-0e20-4a65-b305-f3f50e82e839 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:43:59.008006 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7b6c91-8a76-4f18-9b4d-3ee4e8c4ac6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:43:59.008793 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7b6c91-8a76-4f18-9b4d-3ee4e8c4ac6c admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:59.009165 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7b6c91-8a76-4f18-9b4d-3ee4e8c4ac6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:59.009446 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7b6c91-8a76-4f18-9b4d-3ee4e8c4ac6c admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:59.010212 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7b6c91-8a76-4f18-9b4d-3ee4e8c4ac6c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:43:59.012268 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7b6c91-8a76-4f18-9b4d-3ee4e8c4ac6c admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:43:59.121457 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1381/2806] 10.222.0.66 () {60 vars in 1143 bytes} [Mon Jul 18 02:43:58 2022] POST /identity/v3/users => generated 301 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:43:59.125927 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d58b47-03d0-4f30-9495-09edc177bf9a admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:43:59.149132 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d58b47-03d0-4f30-9495-09edc177bf9a admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:43:59.163916 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b134e0f7-1bce-4b8f-a559-ea2aa4191dba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:43:59.164719 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b134e0f7-1bce-4b8f-a559-ea2aa4191dba admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:59.165105 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b134e0f7-1bce-4b8f-a559-ea2aa4191dba admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:59.166993 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b134e0f7-1bce-4b8f-a559-ea2aa4191dba admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:59.167851 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b134e0f7-1bce-4b8f-a559-ea2aa4191dba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:43:59.177367 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b134e0f7-1bce-4b8f-a559-ea2aa4191dba admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:43:59.190483 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1426/2807] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:43:59 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 02:43:59.194485 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7b6c91-8a76-4f18-9b4d-3ee4e8c4ac6c admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:43:59.205407 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7b6c91-8a76-4f18-9b4d-3ee4e8c4ac6c admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:43:59.209919 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c8a205-ca84-49b8-ad57-9f54e7c23585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:43:59.210913 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c8a205-ca84-49b8-ad57-9f54e7c23585 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:59.211377 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c8a205-ca84-49b8-ad57-9f54e7c23585 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:59.211668 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c8a205-ca84-49b8-ad57-9f54e7c23585 admin admin] PATH_INFO: `/v3/projects/5db41ad2234a463cbcfd79c8597c4245/users/d1d13f0263704dd1a0a6cc0d19d860ea/roles/d4aa3324f0c84887b41d0dc4acf7f30e` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:59.228981 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c8a205-ca84-49b8-ad57-9f54e7c23585 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5db41ad2234a463cbcfd79c8597c4245, user_id=d1d13f0263704dd1a0a6cc0d19d860ea, role_id=d4aa3324f0c84887b41d0dc4acf7f30e)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:43:59.231745 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c8a205-ca84-49b8-ad57-9f54e7c23585 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:43:59.315994 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1382/2808] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:43:59 2022] PUT /identity/v3/projects/5db41ad2234a463cbcfd79c8597c4245/users/d1d13f0263704dd1a0a6cc0d19d860ea/roles/d4aa3324f0c84887b41d0dc4acf7f30e => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:43:59.320045 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b134e0f7-1bce-4b8f-a559-ea2aa4191dba admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:43:59.331808 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b134e0f7-1bce-4b8f-a559-ea2aa4191dba admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:43:59.381722 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:INSERT=4,keystone:DELETE=1,keystone:SELECT=95 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:43:59.393988 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29df5a76-cd26-446f-b956-3efd16654484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:43:59.394707 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29df5a76-cd26-446f-b956-3efd16654484 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:59.395613 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29df5a76-cd26-446f-b956-3efd16654484 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:59.395613 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29df5a76-cd26-446f-b956-3efd16654484 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:59.396282 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29df5a76-cd26-446f-b956-3efd16654484 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:43:59.398066 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29df5a76-cd26-446f-b956-3efd16654484 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:43:59.413743 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1427/2809] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:43:59 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:43:59.415147 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c8a205-ca84-49b8-ad57-9f54e7c23585 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:43:59.431248 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c8a205-ca84-49b8-ad57-9f54e7c23585 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:43:59.457319 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cb6fb0-2dd2-476c-b707-8bfff30e4dc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:43:59.460485 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cb6fb0-2dd2-476c-b707-8bfff30e4dc1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:59.460485 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cb6fb0-2dd2-476c-b707-8bfff30e4dc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:59.460485 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cb6fb0-2dd2-476c-b707-8bfff30e4dc1 admin admin] PATH_INFO: `/v3/projects/5db41ad2234a463cbcfd79c8597c4245/users/d1d13f0263704dd1a0a6cc0d19d860ea/roles/d4aa3324f0c84887b41d0dc4acf7f30e` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:59.460887 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cb6fb0-2dd2-476c-b707-8bfff30e4dc1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5db41ad2234a463cbcfd79c8597c4245, user_id=d1d13f0263704dd1a0a6cc0d19d860ea, role_id=d4aa3324f0c84887b41d0dc4acf7f30e)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:43:59.463643 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cb6fb0-2dd2-476c-b707-8bfff30e4dc1 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:43:59.482045 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1383/2810] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:43:59 2022] PUT /identity/v3/projects/5db41ad2234a463cbcfd79c8597c4245/users/d1d13f0263704dd1a0a6cc0d19d860ea/roles/d4aa3324f0c84887b41d0dc4acf7f30e => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:43:59.552610 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144197d5-1675-457b-ac36-75694e23347d None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:59.553007 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144197d5-1675-457b-ac36-75694e23347d None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:59.553249 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144197d5-1675-457b-ac36-75694e23347d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:59.584987 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7da6a9-18d1-4c23-b0f0-0f3646b4ca91 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:59.585302 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7da6a9-18d1-4c23-b0f0-0f3646b4ca91 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:59.587584 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7da6a9-18d1-4c23-b0f0-0f3646b4ca91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:59.631565 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-144197d5-1675-457b-ac36-75694e23347d None None] MFA Rules not processed for user `d1d13f0263704dd1a0a6cc0d19d860ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:43:59.634790 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-144197d5-1675-457b-ac36-75694e23347d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:43:59.641318 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-ab7da6a9-18d1-4c23-b0f0-0f3646b4ca91 None None] MFA Rules not processed for user `b065087a79524114b5cacb8c5fd3254f`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:43:59.642915 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-ab7da6a9-18d1-4c23-b0f0-0f3646b4ca91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:43:59.743940 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:43:59.763426 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1428/2811] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:43:59 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:43:59.776333 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144197d5-1675-457b-ac36-75694e23347d None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:43:59.778016 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-144197d5-1675-457b-ac36-75694e23347d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:43:59.780337 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1384/2812] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:43:59 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:43:59.793894 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7da6a9-18d1-4c23-b0f0-0f3646b4ca91 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:43:59.800651 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-ab7da6a9-18d1-4c23-b0f0-0f3646b4ca91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:43:59.868222 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144197d5-1675-457b-ac36-75694e23347d None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:43:59.871948 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398be4f8-22f8-404e-b458-2ca7228be9ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd49388facdf446885e55b45ef91f01e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:43:59.872599 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398be4f8-22f8-404e-b458-2ca7228be9ef service cinder] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:59.872881 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398be4f8-22f8-404e-b458-2ca7228be9ef service cinder] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:59.873240 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398be4f8-22f8-404e-b458-2ca7228be9ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:59.879020 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7da6a9-18d1-4c23-b0f0-0f3646b4ca91 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:43:59.882910 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398be4f8-22f8-404e-b458-2ca7228be9ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:43:59.883761 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14e6b82-c977-4ba5-bf3c-28a710d190cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd49388facdf446885e55b45ef91f01e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:43:59.884349 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398be4f8-22f8-404e-b458-2ca7228be9ef service cinder] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:43:59.884488 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14e6b82-c977-4ba5-bf3c-28a710d190cf service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:43:59.884719 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14e6b82-c977-4ba5-bf3c-28a710d190cf service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:43:59.884986 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14e6b82-c977-4ba5-bf3c-28a710d190cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:43:59.894694 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1429/2813] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:43:59 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:43:59.906108 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14e6b82-c977-4ba5-bf3c-28a710d190cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:43:59.909254 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14e6b82-c977-4ba5-bf3c-28a710d190cf service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:43:59.919923 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1385/2814] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:43:59 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:44:00.305135 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398be4f8-22f8-404e-b458-2ca7228be9ef service cinder] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:44:00.395268 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398be4f8-22f8-404e-b458-2ca7228be9ef service cinder] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:44:00.399757 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8feaf3b-21c0-49d8-93a5-5e61f937d857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:44:00.400619 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8feaf3b-21c0-49d8-93a5-5e61f937d857 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:44:00.400903 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8feaf3b-21c0-49d8-93a5-5e61f937d857 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:44:00.401176 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8feaf3b-21c0-49d8-93a5-5e61f937d857 admin admin] PATH_INFO: `/v3/users/e32ea64f74354ef3a5f14937b1a053bf` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:44:00.402414 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8feaf3b-21c0-49d8-93a5-5e61f937d857 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e32ea64f74354ef3a5f14937b1a053bf)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:44:00.410145 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8feaf3b-21c0-49d8-93a5-5e61f937d857 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:44:00.441271 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:44:00.647613 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-f8feaf3b-21c0-49d8-93a5-5e61f937d857 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e32ea64f74354ef3a5f14937b1a053bf'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:44:00.800563 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-f8feaf3b-21c0-49d8-93a5-5e61f937d857 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e32ea64f74354ef3a5f14937b1a053bf'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:44:00.815391 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-f8feaf3b-21c0-49d8-93a5-5e61f937d857 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e32ea64f74354ef3a5f14937b1a053bf'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:44:00.818497 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-f8feaf3b-21c0-49d8-93a5-5e61f937d857 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e32ea64f74354ef3a5f14937b1a053bf'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:44:00.842495 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-f8feaf3b-21c0-49d8-93a5-5e61f937d857 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e32ea64f74354ef3a5f14937b1a053bf'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:44:00.859253 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa05823-ce1f-4922-922b-869317f99168 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:44:00.860236 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa05823-ce1f-4922-922b-869317f99168 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:44:00.861328 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa05823-ce1f-4922-922b-869317f99168 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:44:00.863240 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1430/2815] 10.222.0.66 () {58 vars in 1223 bytes} [Mon Jul 18 02:44:00 2022] DELETE /identity/v3/users/e32ea64f74354ef3a5f14937b1a053bf => generated 0 bytes in 549 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:44:00.888746 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-3aa05823-ce1f-4922-922b-869317f99168 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:44:00.890035 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-3aa05823-ce1f-4922-922b-869317f99168 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:44:00.967779 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1386/2816] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:44:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:44:00.995223 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8feaf3b-21c0-49d8-93a5-5e61f937d857 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:44:00.997923 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-f8feaf3b-21c0-49d8-93a5-5e61f937d857 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:44:01.090997 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8feaf3b-21c0-49d8-93a5-5e61f937d857 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:44:01.095597 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0a605c-c3ed-47c6-8b52-db3c6e40b252 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eaf19a259c2741cea375a38763e33cdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:44:01.096269 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0a605c-c3ed-47c6-8b52-db3c6e40b252 service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:44:01.096553 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0a605c-c3ed-47c6-8b52-db3c6e40b252 service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:44:01.096831 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0a605c-c3ed-47c6-8b52-db3c6e40b252 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:44:01.107088 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0a605c-c3ed-47c6-8b52-db3c6e40b252 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:44:01.109540 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0a605c-c3ed-47c6-8b52-db3c6e40b252 service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:44:01.121355 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1431/2817] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:44:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:44:01.777998 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa05823-ce1f-4922-922b-869317f99168 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:44:01.779250 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-3aa05823-ce1f-4922-922b-869317f99168 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:44:01.794487 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa05823-ce1f-4922-922b-869317f99168 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:44:01.800003 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358af2ef-4f6f-42f2-80dc-67a85db2e726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:44:01.800890 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358af2ef-4f6f-42f2-80dc-67a85db2e726 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:44:01.801204 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358af2ef-4f6f-42f2-80dc-67a85db2e726 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:44:01.801552 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358af2ef-4f6f-42f2-80dc-67a85db2e726 admin admin] PATH_INFO: `/v3/projects/9a43c7e0e2934215b5000d957a1ddc40` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:44:01.802839 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358af2ef-4f6f-42f2-80dc-67a85db2e726 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a43c7e0e2934215b5000d957a1ddc40)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:44:01.804906 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358af2ef-4f6f-42f2-80dc-67a85db2e726 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:44:01.898895 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9304212e-1fbc-42ef-ac6e-979a7271ae0a None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:44:01.899462 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9304212e-1fbc-42ef-ac6e-979a7271ae0a None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:44:01.899462 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9304212e-1fbc-42ef-ac6e-979a7271ae0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:44:01.916801 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-9304212e-1fbc-42ef-ac6e-979a7271ae0a None None] MFA Rules not processed for user `b065087a79524114b5cacb8c5fd3254f`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:44:01.918155 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-9304212e-1fbc-42ef-ac6e-979a7271ae0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:44:01.966620 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-358af2ef-4f6f-42f2-80dc-67a85db2e726 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a43c7e0e2934215b5000d957a1ddc40', 'request_id': 'req-358af2ef-4f6f-42f2-80dc-67a85db2e726'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:44:01.974298 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-358af2ef-4f6f-42f2-80dc-67a85db2e726 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a43c7e0e2934215b5000d957a1ddc40', 'request_id': 'req-358af2ef-4f6f-42f2-80dc-67a85db2e726'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:44:01.990052 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1432/2818] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:44:01 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 02:44:01.990294 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-358af2ef-4f6f-42f2-80dc-67a85db2e726 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a43c7e0e2934215b5000d957a1ddc40', 'request_id': 'req-358af2ef-4f6f-42f2-80dc-67a85db2e726'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:44:02.070444 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-358af2ef-4f6f-42f2-80dc-67a85db2e726 admin admin] The token cache is being invalidate because project 9a43c7e0e2934215b5000d957a1ddc40 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 02:44:02.072747 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-358af2ef-4f6f-42f2-80dc-67a85db2e726 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:44:02.072747 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-358af2ef-4f6f-42f2-80dc-67a85db2e726 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:44:02.077219 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1387/2819] 10.222.0.66 () {58 vars in 1232 bytes} [Mon Jul 18 02:44:01 2022] DELETE /identity/v3/projects/9a43c7e0e2934215b5000d957a1ddc40 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:44:02.078630 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9304212e-1fbc-42ef-ac6e-979a7271ae0a None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:44:02.097051 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9304212e-1fbc-42ef-ac6e-979a7271ae0a None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:44:02.144929 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:44:02.162873 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac378ef9-7b3b-489a-b9fd-1dd150ccd9fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:44:02.164057 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac378ef9-7b3b-489a-b9fd-1dd150ccd9fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:44:02.164057 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac378ef9-7b3b-489a-b9fd-1dd150ccd9fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:44:02.164336 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac378ef9-7b3b-489a-b9fd-1dd150ccd9fe admin admin] PATH_INFO: `/v3/users/d1d13f0263704dd1a0a6cc0d19d860ea` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:44:02.165577 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac378ef9-7b3b-489a-b9fd-1dd150ccd9fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1d13f0263704dd1a0a6cc0d19d860ea)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:44:02.167535 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac378ef9-7b3b-489a-b9fd-1dd150ccd9fe admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:44:02.363764 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-ac378ef9-7b3b-489a-b9fd-1dd150ccd9fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1d13f0263704dd1a0a6cc0d19d860ea'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:44:02.433292 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-ac378ef9-7b3b-489a-b9fd-1dd150ccd9fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1d13f0263704dd1a0a6cc0d19d860ea'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:44:02.442791 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-ac378ef9-7b3b-489a-b9fd-1dd150ccd9fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1d13f0263704dd1a0a6cc0d19d860ea'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:44:02.443580 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-ac378ef9-7b3b-489a-b9fd-1dd150ccd9fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1d13f0263704dd1a0a6cc0d19d860ea'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:44:02.456749 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-ac378ef9-7b3b-489a-b9fd-1dd150ccd9fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1d13f0263704dd1a0a6cc0d19d860ea'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:44:02.469407 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1433/2820] 10.222.0.66 () {58 vars in 1223 bytes} [Mon Jul 18 02:44:02 2022] DELETE /identity/v3/users/d1d13f0263704dd1a0a6cc0d19d860ea => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:44:03.040231 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358af2ef-4f6f-42f2-80dc-67a85db2e726 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:44:03.041661 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-358af2ef-4f6f-42f2-80dc-67a85db2e726 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:44:03.057766 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358af2ef-4f6f-42f2-80dc-67a85db2e726 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:44:03.073535 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:44:03.119927 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b9d6d0-957c-43ad-974d-6f5a1fc882fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:44:03.120791 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b9d6d0-957c-43ad-974d-6f5a1fc882fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:44:03.121078 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b9d6d0-957c-43ad-974d-6f5a1fc882fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:44:03.121357 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b9d6d0-957c-43ad-974d-6f5a1fc882fa admin admin] PATH_INFO: `/v3/projects/5db41ad2234a463cbcfd79c8597c4245` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:44:03.122582 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b9d6d0-957c-43ad-974d-6f5a1fc882fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=5db41ad2234a463cbcfd79c8597c4245)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:44:03.125122 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b9d6d0-957c-43ad-974d-6f5a1fc882fa admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:44:03.244658 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-03b9d6d0-957c-43ad-974d-6f5a1fc882fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5db41ad2234a463cbcfd79c8597c4245', 'request_id': 'req-03b9d6d0-957c-43ad-974d-6f5a1fc882fa'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:44:03.250966 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-03b9d6d0-957c-43ad-974d-6f5a1fc882fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5db41ad2234a463cbcfd79c8597c4245', 'request_id': 'req-03b9d6d0-957c-43ad-974d-6f5a1fc882fa'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:44:03.257219 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-03b9d6d0-957c-43ad-974d-6f5a1fc882fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5db41ad2234a463cbcfd79c8597c4245', 'request_id': 'req-03b9d6d0-957c-43ad-974d-6f5a1fc882fa'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:44:03.354019 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-03b9d6d0-957c-43ad-974d-6f5a1fc882fa admin admin] The token cache is being invalidate because project 5db41ad2234a463cbcfd79c8597c4245 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 02:44:03.357177 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-03b9d6d0-957c-43ad-974d-6f5a1fc882fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:44:03.357177 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-03b9d6d0-957c-43ad-974d-6f5a1fc882fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:44:03.357177 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1388/2821] 10.222.0.66 () {58 vars in 1232 bytes} [Mon Jul 18 02:44:03 2022] DELETE /identity/v3/projects/5db41ad2234a463cbcfd79c8597c4245 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:44:03.362373 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e523370-b421-414d-bd14-901d10ddd1db None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:44:03.362890 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e523370-b421-414d-bd14-901d10ddd1db None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:44:03.363030 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e523370-b421-414d-bd14-901d10ddd1db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:44:03.382050 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-0e523370-b421-414d-bd14-901d10ddd1db None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:44:03.383336 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-0e523370-b421-414d-bd14-901d10ddd1db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:44:03.449969 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1434/2822] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:44:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:44:03.457979 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c443bb0a-cf6d-4de5-86a6-ce54b9fb9b80 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:44:03.458305 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c443bb0a-cf6d-4de5-86a6-ce54b9fb9b80 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:44:03.458591 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c443bb0a-cf6d-4de5-86a6-ce54b9fb9b80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:44:03.477917 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-c443bb0a-cf6d-4de5-86a6-ce54b9fb9b80 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:44:03.479160 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-c443bb0a-cf6d-4de5-86a6-ce54b9fb9b80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:44:03.487712 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1389/2823] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:44:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:44:03.491968 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e523370-b421-414d-bd14-901d10ddd1db None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:44:03.502456 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e523370-b421-414d-bd14-901d10ddd1db None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:44:03.507029 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea126df-1573-4de3-876f-b18b81e15163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:44:03.507676 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea126df-1573-4de3-876f-b18b81e15163 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:44:03.507958 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea126df-1573-4de3-876f-b18b81e15163 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:44:03.508233 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea126df-1573-4de3-876f-b18b81e15163 admin admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:44:03.508952 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea126df-1573-4de3-876f-b18b81e15163 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:44:03.509279 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea126df-1573-4de3-876f-b18b81e15163 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:44:03.510905 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea126df-1573-4de3-876f-b18b81e15163 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:44:03.522679 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1435/2824] 10.222.0.66 () {58 vars in 1152 bytes} [Mon Jul 18 02:44:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:44:03.526910 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c443bb0a-cf6d-4de5-86a6-ce54b9fb9b80 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:44:03.536197 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c443bb0a-cf6d-4de5-86a6-ce54b9fb9b80 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:44:03.541155 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25070bf-3b8a-4f65-9d31-ea2ea7632002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:44:03.546692 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25070bf-3b8a-4f65-9d31-ea2ea7632002 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:44:03.546692 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25070bf-3b8a-4f65-9d31-ea2ea7632002 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:44:03.546692 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25070bf-3b8a-4f65-9d31-ea2ea7632002 admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:44:03.546692 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25070bf-3b8a-4f65-9d31-ea2ea7632002 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:44:03.546692 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25070bf-3b8a-4f65-9d31-ea2ea7632002 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:44:03.680828 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1390/2825] 10.222.0.66 () {60 vars in 1152 bytes} [Mon Jul 18 02:44:03 2022] POST /identity/v3/projects => generated 373 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:44:03.682111 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea126df-1573-4de3-876f-b18b81e15163 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:44:03.693666 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea126df-1573-4de3-876f-b18b81e15163 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:44:03.741692 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:44:03.754928 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6512d7-2839-44f0-b2ef-c57def818c60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:44:03.755511 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6512d7-2839-44f0-b2ef-c57def818c60 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:44:03.758758 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6512d7-2839-44f0-b2ef-c57def818c60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:44:03.758758 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6512d7-2839-44f0-b2ef-c57def818c60 admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:44:03.758758 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6512d7-2839-44f0-b2ef-c57def818c60 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:44:03.758758 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6512d7-2839-44f0-b2ef-c57def818c60 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:44:03.909492 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1436/2826] 10.222.0.66 () {60 vars in 1143 bytes} [Mon Jul 18 02:44:03 2022] POST /identity/v3/users => generated 287 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:44:03.913683 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25070bf-3b8a-4f65-9d31-ea2ea7632002 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:44:03.924186 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25070bf-3b8a-4f65-9d31-ea2ea7632002 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:44:03.931447 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66df5cf1-5337-4e53-802b-de5da1535066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:44:03.932166 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66df5cf1-5337-4e53-802b-de5da1535066 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:44:03.932453 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66df5cf1-5337-4e53-802b-de5da1535066 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:44:03.932726 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66df5cf1-5337-4e53-802b-de5da1535066 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:44:03.933451 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66df5cf1-5337-4e53-802b-de5da1535066 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:44:03.935820 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66df5cf1-5337-4e53-802b-de5da1535066 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:44:03.964483 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1391/2827] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:44:03 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:44:03.968533 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6512d7-2839-44f0-b2ef-c57def818c60 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:44:03.989135 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6512d7-2839-44f0-b2ef-c57def818c60 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:44:03.993984 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073311a2-c23b-4c15-b95d-857fab76af4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:44:03.994885 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073311a2-c23b-4c15-b95d-857fab76af4b admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:44:03.995235 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073311a2-c23b-4c15-b95d-857fab76af4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:44:03.995582 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073311a2-c23b-4c15-b95d-857fab76af4b admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:44:03.996507 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073311a2-c23b-4c15-b95d-857fab76af4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:44:03.998249 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073311a2-c23b-4c15-b95d-857fab76af4b admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:44:04.009719 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1437/2828] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:44:03 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:44:04.014005 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66df5cf1-5337-4e53-802b-de5da1535066 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:44:04.025475 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66df5cf1-5337-4e53-802b-de5da1535066 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:44:04.030479 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f192c0-cbd1-4a1c-9d4b-12dfc4f0938d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:44:04.031582 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f192c0-cbd1-4a1c-9d4b-12dfc4f0938d admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:44:04.031933 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f192c0-cbd1-4a1c-9d4b-12dfc4f0938d admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:44:04.032353 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f192c0-cbd1-4a1c-9d4b-12dfc4f0938d admin admin] PATH_INFO: `/v3/projects/0c5f3a65a41f408ca62f1acfb3a4ffee/users/018f9e146cba41ff9555e30695dc324e/roles/65feabf3b27944a8b01a86c8ac608457` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:44:04.047564 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f192c0-cbd1-4a1c-9d4b-12dfc4f0938d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c5f3a65a41f408ca62f1acfb3a4ffee, user_id=018f9e146cba41ff9555e30695dc324e, role_id=65feabf3b27944a8b01a86c8ac608457)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:44:04.050339 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f192c0-cbd1-4a1c-9d4b-12dfc4f0938d admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:44:04.143484 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1392/2829] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:44:04 2022] PUT /identity/v3/projects/0c5f3a65a41f408ca62f1acfb3a4ffee/users/018f9e146cba41ff9555e30695dc324e/roles/65feabf3b27944a8b01a86c8ac608457 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:44:04.148549 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2530737-0f53-4a60-97db-8164f67eb2eb None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:44:04.148785 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2530737-0f53-4a60-97db-8164f67eb2eb None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:44:04.149058 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2530737-0f53-4a60-97db-8164f67eb2eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:44:04.168053 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-c2530737-0f53-4a60-97db-8164f67eb2eb None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:44:04.169363 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-c2530737-0f53-4a60-97db-8164f67eb2eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:44:04.232831 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1438/2830] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:44:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:44:04.243546 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f192c0-cbd1-4a1c-9d4b-12dfc4f0938d admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:44:04.245341 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-b5f192c0-cbd1-4a1c-9d4b-12dfc4f0938d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:44:04.295495 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:44:04.307135 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f192c0-cbd1-4a1c-9d4b-12dfc4f0938d admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:44:04.313159 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e119e615-d298-4ae0-9a29-e3b49820db03 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eaf19a259c2741cea375a38763e33cdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:44:04.313815 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e119e615-d298-4ae0-9a29-e3b49820db03 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:44:04.314124 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e119e615-d298-4ae0-9a29-e3b49820db03 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:44:04.314663 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e119e615-d298-4ae0-9a29-e3b49820db03 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:44:04.323907 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e119e615-d298-4ae0-9a29-e3b49820db03 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:44:04.325350 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e119e615-d298-4ae0-9a29-e3b49820db03 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:44:04.335305 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1393/2831] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:44:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:44:04.821237 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2530737-0f53-4a60-97db-8164f67eb2eb None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:44:04.822916 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-c2530737-0f53-4a60-97db-8164f67eb2eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:44:04.836978 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2530737-0f53-4a60-97db-8164f67eb2eb None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:44:04.840531 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93510be-6c90-4bd0-9ed5-09f20494aa49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eaf19a259c2741cea375a38763e33cdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:44:04.841165 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93510be-6c90-4bd0-9ed5-09f20494aa49 service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:44:04.841442 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93510be-6c90-4bd0-9ed5-09f20494aa49 service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:44:04.841710 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93510be-6c90-4bd0-9ed5-09f20494aa49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:44:04.844242 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-d93510be-6c90-4bd0-9ed5-09f20494aa49 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:44:04.907345 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93510be-6c90-4bd0-9ed5-09f20494aa49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:44:04.908784 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93510be-6c90-4bd0-9ed5-09f20494aa49 service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:44:04.918413 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1439/2832] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:44:04 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:44:06.367738 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eb7daf-0721-4e90-8a55-03a1fffd21ed None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:44:06.368300 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eb7daf-0721-4e90-8a55-03a1fffd21ed None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:44:06.368697 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eb7daf-0721-4e90-8a55-03a1fffd21ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:44:06.406767 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-16eb7daf-0721-4e90-8a55-03a1fffd21ed None None] MFA Rules not processed for user `018f9e146cba41ff9555e30695dc324e`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:44:06.410014 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-16eb7daf-0721-4e90-8a55-03a1fffd21ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:44:06.492932 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1394/2833] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:44:06 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:44:06.497570 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93510be-6c90-4bd0-9ed5-09f20494aa49 service neutron] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:44:06.509260 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93510be-6c90-4bd0-9ed5-09f20494aa49 service neutron] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:44:06.517231 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724b6a6a-cea2-4c68-b3db-7d12b3e11eb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:44:06.517231 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724b6a6a-cea2-4c68-b3db-7d12b3e11eb9 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:44:06.517231 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724b6a6a-cea2-4c68-b3db-7d12b3e11eb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:44:06.517231 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724b6a6a-cea2-4c68-b3db-7d12b3e11eb9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:44:06.517651 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724b6a6a-cea2-4c68-b3db-7d12b3e11eb9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:44:06.519104 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724b6a6a-cea2-4c68-b3db-7d12b3e11eb9 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:44:06.650310 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1440/2834] 10.222.0.66 () {60 vars in 1152 bytes} [Mon Jul 18 02:44:06 2022] POST /identity/v3/projects => generated 373 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:44:06.651667 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16eb7daf-0721-4e90-8a55-03a1fffd21ed None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:44:06.663219 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16eb7daf-0721-4e90-8a55-03a1fffd21ed None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:44:06.724481 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc495c6-f726-4ece-8747-888398ef5947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:44:06.725212 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc495c6-f726-4ece-8747-888398ef5947 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:44:06.725534 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc495c6-f726-4ece-8747-888398ef5947 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:44:06.725812 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc495c6-f726-4ece-8747-888398ef5947 admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:44:06.726543 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc495c6-f726-4ece-8747-888398ef5947 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:44:06.728541 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc495c6-f726-4ece-8747-888398ef5947 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:44:06.833907 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1395/2835] 10.222.0.66 () {60 vars in 1143 bytes} [Mon Jul 18 02:44:06 2022] POST /identity/v3/users => generated 293 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:44:06.835142 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724b6a6a-cea2-4c68-b3db-7d12b3e11eb9 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:44:06.851816 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724b6a6a-cea2-4c68-b3db-7d12b3e11eb9 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:44:06.860323 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e407116-62a7-4fbe-a825-0d45b7eb24a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:44:06.863460 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e407116-62a7-4fbe-a825-0d45b7eb24a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:44:06.863460 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e407116-62a7-4fbe-a825-0d45b7eb24a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:44:06.863460 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e407116-62a7-4fbe-a825-0d45b7eb24a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:44:06.863460 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e407116-62a7-4fbe-a825-0d45b7eb24a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:44:06.863868 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e407116-62a7-4fbe-a825-0d45b7eb24a4 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:44:06.876383 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1441/2836] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:44:06 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:44:06.880920 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc495c6-f726-4ece-8747-888398ef5947 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:44:06.891943 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc495c6-f726-4ece-8747-888398ef5947 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:44:06.896620 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a92632b-a588-4246-b904-c8887b9c3b91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:44:06.897785 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a92632b-a588-4246-b904-c8887b9c3b91 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:44:06.897785 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a92632b-a588-4246-b904-c8887b9c3b91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:44:06.898064 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a92632b-a588-4246-b904-c8887b9c3b91 admin admin] PATH_INFO: `/v3/projects/9351e7bbd4cc4fefa7c21aaf12199f87/users/ab273eff8b16488ea2f40b1ee5a562f5/roles/d4aa3324f0c84887b41d0dc4acf7f30e` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:44:06.915412 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a92632b-a588-4246-b904-c8887b9c3b91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9351e7bbd4cc4fefa7c21aaf12199f87, user_id=ab273eff8b16488ea2f40b1ee5a562f5, role_id=d4aa3324f0c84887b41d0dc4acf7f30e)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:44:06.918498 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a92632b-a588-4246-b904-c8887b9c3b91 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:44:06.990365 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1396/2837] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:44:06 2022] PUT /identity/v3/projects/9351e7bbd4cc4fefa7c21aaf12199f87/users/ab273eff8b16488ea2f40b1ee5a562f5/roles/d4aa3324f0c84887b41d0dc4acf7f30e => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:44:06.995913 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e407116-62a7-4fbe-a825-0d45b7eb24a4 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:44:07.006813 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e407116-62a7-4fbe-a825-0d45b7eb24a4 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:44:07.036020 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:44:07.067164 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c70614-4c58-4188-aad1-4e5c170a0e0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:44:07.068101 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c70614-4c58-4188-aad1-4e5c170a0e0f admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:44:07.068388 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c70614-4c58-4188-aad1-4e5c170a0e0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:44:07.068666 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c70614-4c58-4188-aad1-4e5c170a0e0f admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:44:07.069394 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c70614-4c58-4188-aad1-4e5c170a0e0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:44:07.071036 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c70614-4c58-4188-aad1-4e5c170a0e0f admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:44:07.081664 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1442/2838] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:44:06 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:44:07.086256 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a92632b-a588-4246-b904-c8887b9c3b91 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:44:07.098237 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a92632b-a588-4246-b904-c8887b9c3b91 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:44:07.102945 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243a0baf-0c04-499a-85f8-c9c02d81226b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:44:07.103837 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243a0baf-0c04-499a-85f8-c9c02d81226b admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:44:07.104116 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243a0baf-0c04-499a-85f8-c9c02d81226b admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:44:07.104392 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243a0baf-0c04-499a-85f8-c9c02d81226b admin admin] PATH_INFO: `/v3/projects/9351e7bbd4cc4fefa7c21aaf12199f87/users/ab273eff8b16488ea2f40b1ee5a562f5/roles/d4aa3324f0c84887b41d0dc4acf7f30e` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:44:07.106396 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243a0baf-0c04-499a-85f8-c9c02d81226b admin admin] RBAC: Authorizing `identity:create_grant(project_id=9351e7bbd4cc4fefa7c21aaf12199f87, user_id=ab273eff8b16488ea2f40b1ee5a562f5, role_id=d4aa3324f0c84887b41d0dc4acf7f30e)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:44:07.109699 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243a0baf-0c04-499a-85f8-c9c02d81226b admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:44:07.120992 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1397/2839] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:44:07 2022] PUT /identity/v3/projects/9351e7bbd4cc4fefa7c21aaf12199f87/users/ab273eff8b16488ea2f40b1ee5a562f5/roles/d4aa3324f0c84887b41d0dc4acf7f30e => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:44:09.112294 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1926d8e9-04fd-441f-b418-e551a04698de None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:44:09.112809 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1926d8e9-04fd-441f-b418-e551a04698de None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:44:09.112911 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1926d8e9-04fd-441f-b418-e551a04698de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:44:09.156757 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-1926d8e9-04fd-441f-b418-e551a04698de None None] MFA Rules not processed for user `ab273eff8b16488ea2f40b1ee5a562f5`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:44:09.158712 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-1926d8e9-04fd-441f-b418-e551a04698de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:44:09.239603 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1443/2840] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:44:09 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:44:09.256429 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243a0baf-0c04-499a-85f8-c9c02d81226b admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:44:09.259801 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-243a0baf-0c04-499a-85f8-c9c02d81226b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:44:09.302672 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:44:09.344074 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243a0baf-0c04-499a-85f8-c9c02d81226b admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:44:09.348380 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9aaafa-3e2c-43ef-9a5e-a26986d53792 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd49388facdf446885e55b45ef91f01e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:44:09.349140 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9aaafa-3e2c-43ef-9a5e-a26986d53792 service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:44:09.349493 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9aaafa-3e2c-43ef-9a5e-a26986d53792 service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:44:09.349837 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9aaafa-3e2c-43ef-9a5e-a26986d53792 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:44:09.425575 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9aaafa-3e2c-43ef-9a5e-a26986d53792 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:44:09.427579 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9aaafa-3e2c-43ef-9a5e-a26986d53792 service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:44:09.445517 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1398/2841] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:44:09 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:44:11.226888 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1926d8e9-04fd-441f-b418-e551a04698de None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:44:11.228283 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-1926d8e9-04fd-441f-b418-e551a04698de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:44:11.241323 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1926d8e9-04fd-441f-b418-e551a04698de None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:44:11.245627 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab09d0c4-ac58-4569-af5e-c2e2c4985adf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd49388facdf446885e55b45ef91f01e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:44:11.246330 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab09d0c4-ac58-4569-af5e-c2e2c4985adf service cinder] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:44:11.246617 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab09d0c4-ac58-4569-af5e-c2e2c4985adf service cinder] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:44:11.246899 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab09d0c4-ac58-4569-af5e-c2e2c4985adf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:44:11.256287 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab09d0c4-ac58-4569-af5e-c2e2c4985adf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:44:11.257729 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab09d0c4-ac58-4569-af5e-c2e2c4985adf service cinder] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:44:11.267408 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1444/2842] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:44:11 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:44:14.635372 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9aaafa-3e2c-43ef-9a5e-a26986d53792 service cinder] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:44:14.727118 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9aaafa-3e2c-43ef-9a5e-a26986d53792 service cinder] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:44:14.732486 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be321b20-adb3-4e6e-8306-62d7ca8cae1f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eaf19a259c2741cea375a38763e33cdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:44:14.734127 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be321b20-adb3-4e6e-8306-62d7ca8cae1f service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:44:14.734280 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be321b20-adb3-4e6e-8306-62d7ca8cae1f service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:44:14.734604 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be321b20-adb3-4e6e-8306-62d7ca8cae1f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:44:14.736868 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-be321b20-adb3-4e6e-8306-62d7ca8cae1f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:44:14.809511 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be321b20-adb3-4e6e-8306-62d7ca8cae1f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:44:14.811000 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be321b20-adb3-4e6e-8306-62d7ca8cae1f service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:44:14.822063 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1399/2843] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:44:14 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:44:21.263017 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=56 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:44:24.816797 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=85 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:44:27.538960 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab09d0c4-ac58-4569-af5e-c2e2c4985adf service cinder] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:44:27.608091 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab09d0c4-ac58-4569-af5e-c2e2c4985adf service cinder] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:44:27.612968 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8fdacc-dedd-4c68-acda-4a2ac8924b83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:44:27.614263 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8fdacc-dedd-4c68-acda-4a2ac8924b83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:44:27.614502 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8fdacc-dedd-4c68-acda-4a2ac8924b83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:44:27.614783 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8fdacc-dedd-4c68-acda-4a2ac8924b83 admin admin] PATH_INFO: `/v3/users/018f9e146cba41ff9555e30695dc324e` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:44:27.616275 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8fdacc-dedd-4c68-acda-4a2ac8924b83 admin admin] RBAC: Authorizing `identity:delete_user(user_id=018f9e146cba41ff9555e30695dc324e)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:44:27.618552 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8fdacc-dedd-4c68-acda-4a2ac8924b83 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:44:27.837533 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-3c8fdacc-dedd-4c68-acda-4a2ac8924b83 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '018f9e146cba41ff9555e30695dc324e'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:44:27.914556 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-3c8fdacc-dedd-4c68-acda-4a2ac8924b83 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '018f9e146cba41ff9555e30695dc324e'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:44:27.922162 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-3c8fdacc-dedd-4c68-acda-4a2ac8924b83 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '018f9e146cba41ff9555e30695dc324e'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:44:27.922811 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-3c8fdacc-dedd-4c68-acda-4a2ac8924b83 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '018f9e146cba41ff9555e30695dc324e'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:44:27.934846 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-3c8fdacc-dedd-4c68-acda-4a2ac8924b83 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '018f9e146cba41ff9555e30695dc324e'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:44:27.941122 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1445/2844] 10.222.0.66 () {58 vars in 1223 bytes} [Mon Jul 18 02:44:27 2022] DELETE /identity/v3/users/018f9e146cba41ff9555e30695dc324e => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:44:28.213228 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be321b20-adb3-4e6e-8306-62d7ca8cae1f service neutron] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:44:28.223924 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be321b20-adb3-4e6e-8306-62d7ca8cae1f service neutron] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:44:28.290708 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eef0935-2389-4a51-84bf-83047a8e32bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:44:28.291637 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eef0935-2389-4a51-84bf-83047a8e32bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:44:28.291863 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eef0935-2389-4a51-84bf-83047a8e32bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:44:28.292135 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eef0935-2389-4a51-84bf-83047a8e32bc admin admin] PATH_INFO: `/v3/projects/0c5f3a65a41f408ca62f1acfb3a4ffee` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:44:28.293309 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eef0935-2389-4a51-84bf-83047a8e32bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c5f3a65a41f408ca62f1acfb3a4ffee)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:44:28.295262 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eef0935-2389-4a51-84bf-83047a8e32bc admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:44:28.412713 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-8eef0935-2389-4a51-84bf-83047a8e32bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c5f3a65a41f408ca62f1acfb3a4ffee', 'request_id': 'req-8eef0935-2389-4a51-84bf-83047a8e32bc'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:44:28.418827 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-8eef0935-2389-4a51-84bf-83047a8e32bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c5f3a65a41f408ca62f1acfb3a4ffee', 'request_id': 'req-8eef0935-2389-4a51-84bf-83047a8e32bc'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:44:28.425057 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-8eef0935-2389-4a51-84bf-83047a8e32bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c5f3a65a41f408ca62f1acfb3a4ffee', 'request_id': 'req-8eef0935-2389-4a51-84bf-83047a8e32bc'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:44:28.480543 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-8eef0935-2389-4a51-84bf-83047a8e32bc admin admin] The token cache is being invalidate because project 0c5f3a65a41f408ca62f1acfb3a4ffee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 02:44:28.480841 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-8eef0935-2389-4a51-84bf-83047a8e32bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:44:28.481478 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-8eef0935-2389-4a51-84bf-83047a8e32bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:44:28.483215 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1400/2845] 10.222.0.66 () {58 vars in 1232 bytes} [Mon Jul 18 02:44:28 2022] DELETE /identity/v3/projects/0c5f3a65a41f408ca62f1acfb3a4ffee => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:44:28.488296 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8fdacc-dedd-4c68-acda-4a2ac8924b83 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:44:28.489880 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-3c8fdacc-dedd-4c68-acda-4a2ac8924b83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:44:28.506816 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8fdacc-dedd-4c68-acda-4a2ac8924b83 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:44:28.567553 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b8fc92-c001-4851-b179-b8fb907190e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:44:28.568507 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b8fc92-c001-4851-b179-b8fb907190e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:44:28.568987 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b8fc92-c001-4851-b179-b8fb907190e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:44:28.569347 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b8fc92-c001-4851-b179-b8fb907190e2 admin admin] PATH_INFO: `/v3/users/ab273eff8b16488ea2f40b1ee5a562f5` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:44:28.570603 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b8fc92-c001-4851-b179-b8fb907190e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab273eff8b16488ea2f40b1ee5a562f5)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:44:28.572599 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b8fc92-c001-4851-b179-b8fb907190e2 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:44:28.593881 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:44:28.797559 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-a6b8fc92-c001-4851-b179-b8fb907190e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab273eff8b16488ea2f40b1ee5a562f5'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:44:28.880278 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-a6b8fc92-c001-4851-b179-b8fb907190e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab273eff8b16488ea2f40b1ee5a562f5'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:44:28.888073 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-a6b8fc92-c001-4851-b179-b8fb907190e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab273eff8b16488ea2f40b1ee5a562f5'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:44:28.888781 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-a6b8fc92-c001-4851-b179-b8fb907190e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab273eff8b16488ea2f40b1ee5a562f5'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:44:28.902615 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-a6b8fc92-c001-4851-b179-b8fb907190e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab273eff8b16488ea2f40b1ee5a562f5'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:44:28.908953 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1446/2846] 10.222.0.66 () {58 vars in 1223 bytes} [Mon Jul 18 02:44:28 2022] DELETE /identity/v3/users/ab273eff8b16488ea2f40b1ee5a562f5 => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:44:29.156198 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eef0935-2389-4a51-84bf-83047a8e32bc admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:44:29.157644 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-8eef0935-2389-4a51-84bf-83047a8e32bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:44:29.175424 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eef0935-2389-4a51-84bf-83047a8e32bc admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:44:29.236520 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc290eb-5645-44df-ba94-273da6d2dcb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:44:29.237379 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc290eb-5645-44df-ba94-273da6d2dcb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:44:29.237662 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc290eb-5645-44df-ba94-273da6d2dcb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:44:29.237942 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc290eb-5645-44df-ba94-273da6d2dcb4 admin admin] PATH_INFO: `/v3/projects/9351e7bbd4cc4fefa7c21aaf12199f87` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:44:29.239236 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc290eb-5645-44df-ba94-273da6d2dcb4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9351e7bbd4cc4fefa7c21aaf12199f87)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:44:29.241215 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc290eb-5645-44df-ba94-273da6d2dcb4 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:44:29.383822 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:44:29.395046 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-2bc290eb-5645-44df-ba94-273da6d2dcb4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9351e7bbd4cc4fefa7c21aaf12199f87', 'request_id': 'req-2bc290eb-5645-44df-ba94-273da6d2dcb4'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:44:29.400522 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-2bc290eb-5645-44df-ba94-273da6d2dcb4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9351e7bbd4cc4fefa7c21aaf12199f87', 'request_id': 'req-2bc290eb-5645-44df-ba94-273da6d2dcb4'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:44:29.406524 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-2bc290eb-5645-44df-ba94-273da6d2dcb4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9351e7bbd4cc4fefa7c21aaf12199f87', 'request_id': 'req-2bc290eb-5645-44df-ba94-273da6d2dcb4'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:44:29.491027 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-2bc290eb-5645-44df-ba94-273da6d2dcb4 admin admin] The token cache is being invalidate because project 9351e7bbd4cc4fefa7c21aaf12199f87 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 02:44:29.491317 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-2bc290eb-5645-44df-ba94-273da6d2dcb4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:44:29.492428 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-2bc290eb-5645-44df-ba94-273da6d2dcb4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:44:29.494043 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1401/2847] 10.222.0.66 () {58 vars in 1232 bytes} [Mon Jul 18 02:44:29 2022] DELETE /identity/v3/projects/9351e7bbd4cc4fefa7c21aaf12199f87 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:44:29.500113 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c536bd57-0cde-4080-8884-d77452a1330f None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:44:29.500493 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c536bd57-0cde-4080-8884-d77452a1330f None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:44:29.500843 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c536bd57-0cde-4080-8884-d77452a1330f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:44:29.522596 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-c536bd57-0cde-4080-8884-d77452a1330f None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:44:29.524050 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-c536bd57-0cde-4080-8884-d77452a1330f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:44:29.594846 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1447/2848] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:44:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:44:29.602957 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d055af-1d75-4297-9ceb-6a3d526a617c None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:44:29.603286 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d055af-1d75-4297-9ceb-6a3d526a617c None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:44:29.603539 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d055af-1d75-4297-9ceb-6a3d526a617c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:44:29.624703 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-05d055af-1d75-4297-9ceb-6a3d526a617c None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:44:29.625939 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-05d055af-1d75-4297-9ceb-6a3d526a617c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:44:29.634501 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1402/2849] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:44:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:44:29.638852 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c536bd57-0cde-4080-8884-d77452a1330f None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:44:29.648835 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c536bd57-0cde-4080-8884-d77452a1330f None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:44:29.653603 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc62fca3-caf4-4868-8ee7-a51c952294c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:44:29.654307 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc62fca3-caf4-4868-8ee7-a51c952294c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:44:29.654610 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc62fca3-caf4-4868-8ee7-a51c952294c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:44:29.654888 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc62fca3-caf4-4868-8ee7-a51c952294c7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:44:29.655660 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc62fca3-caf4-4868-8ee7-a51c952294c7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 02:44:29.656052 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc62fca3-caf4-4868-8ee7-a51c952294c7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:44:29.657638 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc62fca3-caf4-4868-8ee7-a51c952294c7 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:44:29.670314 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1448/2850] 10.222.0.66 () {58 vars in 1152 bytes} [Mon Jul 18 02:44:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 02:44:29.674775 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d055af-1d75-4297-9ceb-6a3d526a617c None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:44:29.685820 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d055af-1d75-4297-9ceb-6a3d526a617c None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:44:29.690739 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda3b26a-50ad-44e2-9d50-e480bdfece9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:44:29.691382 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda3b26a-50ad-44e2-9d50-e480bdfece9c admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:44:29.691661 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda3b26a-50ad-44e2-9d50-e480bdfece9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:44:29.691933 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda3b26a-50ad-44e2-9d50-e480bdfece9c admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:44:29.692619 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda3b26a-50ad-44e2-9d50-e480bdfece9c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:44:29.694555 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda3b26a-50ad-44e2-9d50-e480bdfece9c admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:44:29.784755 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1403/2851] 10.222.0.66 () {60 vars in 1152 bytes} [Mon Jul 18 02:44:29 2022] POST /identity/v3/projects => generated 385 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:44:29.789371 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc62fca3-caf4-4868-8ee7-a51c952294c7 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:44:29.799840 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc62fca3-caf4-4868-8ee7-a51c952294c7 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:44:29.857252 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9ce5f7-288e-4b37-a51b-6d0df91c4e4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:44:29.857924 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9ce5f7-288e-4b37-a51b-6d0df91c4e4e admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:44:29.858238 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9ce5f7-288e-4b37-a51b-6d0df91c4e4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:44:29.858513 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9ce5f7-288e-4b37-a51b-6d0df91c4e4e admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:44:29.859244 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9ce5f7-288e-4b37-a51b-6d0df91c4e4e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:44:29.861208 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9ce5f7-288e-4b37-a51b-6d0df91c4e4e admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:44:29.920973 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1449/2852] 10.222.0.66 () {60 vars in 1143 bytes} [Mon Jul 18 02:44:29 2022] POST /identity/v3/users => generated 293 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 02:44:29.929304 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda3b26a-50ad-44e2-9d50-e480bdfece9c admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:44:29.940860 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda3b26a-50ad-44e2-9d50-e480bdfece9c admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:44:29.945455 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0872dba4-863c-4d18-9037-d47b62c0fcc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:44:29.946135 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0872dba4-863c-4d18-9037-d47b62c0fcc9 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:44:29.946422 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0872dba4-863c-4d18-9037-d47b62c0fcc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:44:29.946699 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0872dba4-863c-4d18-9037-d47b62c0fcc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:44:29.947417 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0872dba4-863c-4d18-9037-d47b62c0fcc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:44:29.949020 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0872dba4-863c-4d18-9037-d47b62c0fcc9 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:44:29.959377 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1404/2853] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:44:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:44:29.963677 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9ce5f7-288e-4b37-a51b-6d0df91c4e4e admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:44:29.977030 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9ce5f7-288e-4b37-a51b-6d0df91c4e4e admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:44:29.981758 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833df5d8-bfc8-4e69-a187-02f57a02073a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:44:29.982460 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833df5d8-bfc8-4e69-a187-02f57a02073a admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:44:29.982744 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833df5d8-bfc8-4e69-a187-02f57a02073a admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:44:29.983017 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833df5d8-bfc8-4e69-a187-02f57a02073a admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:44:29.983741 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833df5d8-bfc8-4e69-a187-02f57a02073a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:44:29.985368 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833df5d8-bfc8-4e69-a187-02f57a02073a admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:44:29.996430 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1450/2854] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:44:29 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 02:44:30.000680 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0872dba4-863c-4d18-9037-d47b62c0fcc9 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:44:30.011271 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0872dba4-863c-4d18-9037-d47b62c0fcc9 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:44:30.015906 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b965175b-da16-4d53-b68d-f19f5e0ecbd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:44:30.016814 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b965175b-da16-4d53-b68d-f19f5e0ecbd0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:44:30.017096 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b965175b-da16-4d53-b68d-f19f5e0ecbd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:44:30.017371 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b965175b-da16-4d53-b68d-f19f5e0ecbd0 admin admin] PATH_INFO: `/v3/projects/73679a05b6dc4583842dacfeb898876f/users/3277db965cea4f23b463836d26990e1c/roles/65feabf3b27944a8b01a86c8ac608457` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:44:30.032708 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b965175b-da16-4d53-b68d-f19f5e0ecbd0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=73679a05b6dc4583842dacfeb898876f, user_id=3277db965cea4f23b463836d26990e1c, role_id=65feabf3b27944a8b01a86c8ac608457)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:44:30.035419 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b965175b-da16-4d53-b68d-f19f5e0ecbd0 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:44:30.103987 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1405/2855] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:44:29 2022] PUT /identity/v3/projects/73679a05b6dc4583842dacfeb898876f/users/3277db965cea4f23b463836d26990e1c/roles/65feabf3b27944a8b01a86c8ac608457 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:44:30.109094 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd20d1a-fb29-4852-af81-c39099bd0502 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:44:30.109389 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd20d1a-fb29-4852-af81-c39099bd0502 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:44:30.109668 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd20d1a-fb29-4852-af81-c39099bd0502 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:44:30.116416 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:44:30.128647 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-0fd20d1a-fb29-4852-af81-c39099bd0502 None None] MFA Rules not processed for user `62dd67591e0147fe97b248079359a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:44:30.129930 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-0fd20d1a-fb29-4852-af81-c39099bd0502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:44:30.192158 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1451/2856] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:44:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:44:30.201776 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b965175b-da16-4d53-b68d-f19f5e0ecbd0 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:44:30.203566 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-b965175b-da16-4d53-b68d-f19f5e0ecbd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:44:30.261040 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b965175b-da16-4d53-b68d-f19f5e0ecbd0 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:44:30.264724 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f30350-7086-4c4e-bfb2-97e216655368 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eaf19a259c2741cea375a38763e33cdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:44:30.265362 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f30350-7086-4c4e-bfb2-97e216655368 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:44:30.265644 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f30350-7086-4c4e-bfb2-97e216655368 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:44:30.265920 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f30350-7086-4c4e-bfb2-97e216655368 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:44:30.275401 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f30350-7086-4c4e-bfb2-97e216655368 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:44:30.276833 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f30350-7086-4c4e-bfb2-97e216655368 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:44:30.286999 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1406/2857] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:44:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:44:35.897816 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7934929f-be4f-4097-8bbb-1e1a65e44a34 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:44:35.897816 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7934929f-be4f-4097-8bbb-1e1a65e44a34 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:44:35.897816 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7934929f-be4f-4097-8bbb-1e1a65e44a34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:44:35.960137 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-7934929f-be4f-4097-8bbb-1e1a65e44a34 None None] MFA Rules not processed for user `3277db965cea4f23b463836d26990e1c`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:44:35.960137 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-7934929f-be4f-4097-8bbb-1e1a65e44a34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:44:36.032174 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1452/2858] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:44:35 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 02:44:36.036753 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f30350-7086-4c4e-bfb2-97e216655368 service neutron] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:44:36.048159 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f30350-7086-4c4e-bfb2-97e216655368 service neutron] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:44:36.053657 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9eb1dc6-d170-4843-b430-1e4d24fdff7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:44:36.054553 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eb1dc6-d170-4843-b430-1e4d24fdff7a admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:44:36.054843 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eb1dc6-d170-4843-b430-1e4d24fdff7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:44:36.055117 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eb1dc6-d170-4843-b430-1e4d24fdff7a admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:44:36.055823 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9eb1dc6-d170-4843-b430-1e4d24fdff7a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:44:36.058198 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9eb1dc6-d170-4843-b430-1e4d24fdff7a admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:44:36.186322 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1407/2859] 10.222.0.66 () {60 vars in 1152 bytes} [Mon Jul 18 02:44:36 2022] POST /identity/v3/projects => generated 383 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:44:36.190677 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7934929f-be4f-4097-8bbb-1e1a65e44a34 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:44:36.203247 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7934929f-be4f-4097-8bbb-1e1a65e44a34 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:44:36.272388 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2d3348-58e7-4d05-ad66-73b8c8f94ebb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:44:36.273106 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2d3348-58e7-4d05-ad66-73b8c8f94ebb admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:44:36.273388 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2d3348-58e7-4d05-ad66-73b8c8f94ebb admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:44:36.273662 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2d3348-58e7-4d05-ad66-73b8c8f94ebb admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:44:36.274385 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2d3348-58e7-4d05-ad66-73b8c8f94ebb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:44:36.276284 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2d3348-58e7-4d05-ad66-73b8c8f94ebb admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:44:36.377222 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1453/2860] 10.222.0.66 () {60 vars in 1143 bytes} [Mon Jul 18 02:44:36 2022] POST /identity/v3/users => generated 298 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 02:44:36.381484 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9eb1dc6-d170-4843-b430-1e4d24fdff7a admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:44:36.392612 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9eb1dc6-d170-4843-b430-1e4d24fdff7a admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:44:36.397516 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d911978-6f30-4580-a47c-4d4001ebd818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:44:36.398218 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d911978-6f30-4580-a47c-4d4001ebd818 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:44:36.398549 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d911978-6f30-4580-a47c-4d4001ebd818 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:44:36.398833 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d911978-6f30-4580-a47c-4d4001ebd818 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:44:36.399712 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d911978-6f30-4580-a47c-4d4001ebd818 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:44:36.401387 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d911978-6f30-4580-a47c-4d4001ebd818 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:44:36.412545 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1408/2861] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:44:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:44:36.416955 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2d3348-58e7-4d05-ad66-73b8c8f94ebb admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:44:36.427270 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2d3348-58e7-4d05-ad66-73b8c8f94ebb admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:44:36.432402 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23749a9e-50cf-434a-8a28-662031a5c468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:44:36.433284 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23749a9e-50cf-434a-8a28-662031a5c468 admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:44:36.433565 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23749a9e-50cf-434a-8a28-662031a5c468 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:44:36.433899 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23749a9e-50cf-434a-8a28-662031a5c468 admin admin] PATH_INFO: `/v3/projects/59d0c1f5793f4ac3aa9567bd99cf987d/users/764dad0085f846a0bc6855b94bf782d5/roles/d4aa3324f0c84887b41d0dc4acf7f30e` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:44:36.449446 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23749a9e-50cf-434a-8a28-662031a5c468 admin admin] RBAC: Authorizing `identity:create_grant(project_id=59d0c1f5793f4ac3aa9567bd99cf987d, user_id=764dad0085f846a0bc6855b94bf782d5, role_id=d4aa3324f0c84887b41d0dc4acf7f30e)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:44:36.452121 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23749a9e-50cf-434a-8a28-662031a5c468 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:44:36.510825 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:44:36.521150 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1454/2862] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:44:36 2022] PUT /identity/v3/projects/59d0c1f5793f4ac3aa9567bd99cf987d/users/764dad0085f846a0bc6855b94bf782d5/roles/d4aa3324f0c84887b41d0dc4acf7f30e => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 02:44:36.521303 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d911978-6f30-4580-a47c-4d4001ebd818 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:44:36.529611 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d911978-6f30-4580-a47c-4d4001ebd818 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:44:36.553560 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:44:36.586786 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6720f1-77c7-4263-8701-6e35775f4c4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:44:36.587463 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6720f1-77c7-4263-8701-6e35775f4c4e admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:44:36.587749 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6720f1-77c7-4263-8701-6e35775f4c4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:44:36.588028 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6720f1-77c7-4263-8701-6e35775f4c4e admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:44:36.588750 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6720f1-77c7-4263-8701-6e35775f4c4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:44:36.590442 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6720f1-77c7-4263-8701-6e35775f4c4e admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:44:36.600173 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1409/2863] 10.222.0.66 () {58 vars in 1121 bytes} [Mon Jul 18 02:44:36 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 02:44:36.604932 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23749a9e-50cf-434a-8a28-662031a5c468 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:44:36.616377 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23749a9e-50cf-434a-8a28-662031a5c468 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:44:36.621844 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40a38f3-e798-4c10-8708-333f8c3369f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:44:36.622757 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40a38f3-e798-4c10-8708-333f8c3369f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:44:36.623039 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40a38f3-e798-4c10-8708-333f8c3369f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:44:36.623312 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40a38f3-e798-4c10-8708-333f8c3369f9 admin admin] PATH_INFO: `/v3/projects/59d0c1f5793f4ac3aa9567bd99cf987d/users/764dad0085f846a0bc6855b94bf782d5/roles/d4aa3324f0c84887b41d0dc4acf7f30e` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:44:36.625566 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40a38f3-e798-4c10-8708-333f8c3369f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=59d0c1f5793f4ac3aa9567bd99cf987d, user_id=764dad0085f846a0bc6855b94bf782d5, role_id=d4aa3324f0c84887b41d0dc4acf7f30e)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:44:36.628486 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40a38f3-e798-4c10-8708-333f8c3369f9 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:44:36.640159 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1455/2864] 10.222.0.66 () {60 vars in 1482 bytes} [Mon Jul 18 02:44:36 2022] PUT /identity/v3/projects/59d0c1f5793f4ac3aa9567bd99cf987d/users/764dad0085f846a0bc6855b94bf782d5/roles/d4aa3324f0c84887b41d0dc4acf7f30e => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:44:42.358203 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a4c3c2-947e-45bf-b2b2-41f320032987 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:44:42.358705 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a4c3c2-947e-45bf-b2b2-41f320032987 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:44:42.358824 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a4c3c2-947e-45bf-b2b2-41f320032987 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:44:42.401484 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-72a4c3c2-947e-45bf-b2b2-41f320032987 None None] MFA Rules not processed for user `764dad0085f846a0bc6855b94bf782d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:44:42.403075 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-72a4c3c2-947e-45bf-b2b2-41f320032987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:44:42.484075 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1410/2865] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:44:42 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:44:42.495974 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40a38f3-e798-4c10-8708-333f8c3369f9 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:44:42.497782 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-d40a38f3-e798-4c10-8708-333f8c3369f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:44:42.558426 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40a38f3-e798-4c10-8708-333f8c3369f9 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:44:42.562062 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fd87da-d167-43fe-b618-bab0dea76554 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1391b81e7234bc18e22b9e84f231256', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:44:42.562745 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fd87da-d167-43fe-b618-bab0dea76554 service glance] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:44:42.563030 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fd87da-d167-43fe-b618-bab0dea76554 service glance] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:44:42.563305 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fd87da-d167-43fe-b618-bab0dea76554 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:44:42.625403 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fd87da-d167-43fe-b618-bab0dea76554 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:44:42.627079 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fd87da-d167-43fe-b618-bab0dea76554 service glance] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:44:42.637059 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1456/2866] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:44:42 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:44:42.902678 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e32bd5-0ae4-4624-88fd-1d4e7cf306c0 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:44:42.903156 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e32bd5-0ae4-4624-88fd-1d4e7cf306c0 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:44:42.903436 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e32bd5-0ae4-4624-88fd-1d4e7cf306c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:44:42.932458 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-32e32bd5-0ae4-4624-88fd-1d4e7cf306c0 None None] MFA Rules not processed for user `768f872096584d4093fc616d547aabb6`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:44:42.933659 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-32e32bd5-0ae4-4624-88fd-1d4e7cf306c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:44:42.989616 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1411/2867] 10.222.0.66 () {58 vars in 1009 bytes} [Mon Jul 18 02:44:42 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:44:43.137101 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fd87da-d167-43fe-b618-bab0dea76554 service glance] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:44:43.138622 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-b6fd87da-d167-43fe-b618-bab0dea76554 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:44:43.209273 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fd87da-d167-43fe-b618-bab0dea76554 service glance] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:44:43.214466 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e521ab3-9585-4e19-812c-0772e69f6cd8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4300622cbd347dd8f5227bb960e98d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:44:43.215125 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e521ab3-9585-4e19-812c-0772e69f6cd8 service swift] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:44:43.215419 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e521ab3-9585-4e19-812c-0772e69f6cd8 service swift] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:44:43.215701 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e521ab3-9585-4e19-812c-0772e69f6cd8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:44:43.224405 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e521ab3-9585-4e19-812c-0772e69f6cd8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:44:43.226067 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e521ab3-9585-4e19-812c-0772e69f6cd8 service swift] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:44:43.234226 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1457/2868] 10.222.0.66 () {60 vars in 1384 bytes} [Mon Jul 18 02:44:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 02:44:52.982974 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=80 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:44:53.231175 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=85,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:44:53.682529 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074f81f7-6226-4bf4-bfe1-5d4f6c482892 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:44:53.682821 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074f81f7-6226-4bf4-bfe1-5d4f6c482892 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:44:53.683105 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074f81f7-6226-4bf4-bfe1-5d4f6c482892 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:44:53.700692 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-074f81f7-6226-4bf4-bfe1-5d4f6c482892 None None] MFA Rules not processed for user `3277db965cea4f23b463836d26990e1c`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:44:53.701920 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-074f81f7-6226-4bf4-bfe1-5d4f6c482892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:44:53.708878 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1412/2869] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:44:53 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:44:59.008012 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d4c0de-96bd-479b-b14a-fa00dfdccab4 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:44:59.008503 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d4c0de-96bd-479b-b14a-fa00dfdccab4 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:44:59.008779 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d4c0de-96bd-479b-b14a-fa00dfdccab4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:44:59.026767 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-52d4c0de-96bd-479b-b14a-fa00dfdccab4 None None] MFA Rules not processed for user `768f872096584d4093fc616d547aabb6`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:44:59.028105 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-52d4c0de-96bd-479b-b14a-fa00dfdccab4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:44:59.035461 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1458/2870] 10.222.0.66 () {58 vars in 1009 bytes} [Mon Jul 18 02:44:59 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:44:59.048231 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074f81f7-6226-4bf4-bfe1-5d4f6c482892 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:44:59.060726 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074f81f7-6226-4bf4-bfe1-5d4f6c482892 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:44:59.066193 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2783b14a-7ad0-4eee-ab86-5e54c21abdbc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4300622cbd347dd8f5227bb960e98d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:44:59.066842 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2783b14a-7ad0-4eee-ab86-5e54c21abdbc service swift] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:44:59.067130 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2783b14a-7ad0-4eee-ab86-5e54c21abdbc service swift] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:44:59.067411 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2783b14a-7ad0-4eee-ab86-5e54c21abdbc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:44:59.075925 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2783b14a-7ad0-4eee-ab86-5e54c21abdbc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:44:59.079397 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2783b14a-7ad0-4eee-ab86-5e54c21abdbc service swift] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:44:59.085650 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1413/2871] 10.222.0.66 () {60 vars in 1384 bytes} [Mon Jul 18 02:44:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 02:45:01.006612 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d4c0de-96bd-479b-b14a-fa00dfdccab4 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:45:01.061734 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d4c0de-96bd-479b-b14a-fa00dfdccab4 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:45:01.065745 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6cb16c-eb4c-4bb4-bf2f-5550eca103dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eaf19a259c2741cea375a38763e33cdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:45:01.066412 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6cb16c-eb4c-4bb4-bf2f-5550eca103dd service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:45:01.066695 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6cb16c-eb4c-4bb4-bf2f-5550eca103dd service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:45:01.067019 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6cb16c-eb4c-4bb4-bf2f-5550eca103dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:45:01.068788 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-9b6cb16c-eb4c-4bb4-bf2f-5550eca103dd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:45:01.138387 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6cb16c-eb4c-4bb4-bf2f-5550eca103dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:45:01.139826 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6cb16c-eb4c-4bb4-bf2f-5550eca103dd service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:45:01.150743 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1459/2872] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:45:01 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:45:02.341118 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2783b14a-7ad0-4eee-ab86-5e54c21abdbc service swift] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:45:02.345045 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-2783b14a-7ad0-4eee-ab86-5e54c21abdbc service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:45:02.356224 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2783b14a-7ad0-4eee-ab86-5e54c21abdbc service swift] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:45:02.359828 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0465ccaf-91c7-4675-9f5f-0b09edd06443 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eaf19a259c2741cea375a38763e33cdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:45:02.360472 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0465ccaf-91c7-4675-9f5f-0b09edd06443 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:45:02.360750 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0465ccaf-91c7-4675-9f5f-0b09edd06443 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:45:02.361025 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0465ccaf-91c7-4675-9f5f-0b09edd06443 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:45:02.362840 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-0465ccaf-91c7-4675-9f5f-0b09edd06443 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:45:02.378716 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0465ccaf-91c7-4675-9f5f-0b09edd06443 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:45:02.381334 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0465ccaf-91c7-4675-9f5f-0b09edd06443 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:45:02.395248 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1414/2873] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:45:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 02:45:11.145943 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=57 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:45:12.389640 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=15 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:45:29.095632 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c775bfd5-6301-4206-aa9d-acd601fbdc17 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:45:29.096148 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c775bfd5-6301-4206-aa9d-acd601fbdc17 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:45:29.096269 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c775bfd5-6301-4206-aa9d-acd601fbdc17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:45:29.113508 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-c775bfd5-6301-4206-aa9d-acd601fbdc17 None None] MFA Rules not processed for user `b065087a79524114b5cacb8c5fd3254f`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:45:29.114732 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-c775bfd5-6301-4206-aa9d-acd601fbdc17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:45:29.169825 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1460/2874] 10.222.0.66 () {58 vars in 957 bytes} [Mon Jul 18 02:45:29 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:45:29.832860 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0465ccaf-91c7-4675-9f5f-0b09edd06443 service neutron] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:45:29.834427 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-0465ccaf-91c7-4675-9f5f-0b09edd06443 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:45:29.914810 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0465ccaf-91c7-4675-9f5f-0b09edd06443 service neutron] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:45:29.919871 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d894a863-ae92-436b-b74a-c682d3a84c65 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88936b31709b429f8b6777c673ed7b8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:45:29.920535 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d894a863-ae92-436b-b74a-c682d3a84c65 service placement] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:45:29.920823 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d894a863-ae92-436b-b74a-c682d3a84c65 service placement] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:45:29.921103 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d894a863-ae92-436b-b74a-c682d3a84c65 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:45:29.923075 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-d894a863-ae92-436b-b74a-c682d3a84c65 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:45:29.936657 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d894a863-ae92-436b-b74a-c682d3a84c65 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:45:29.938310 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d894a863-ae92-436b-b74a-c682d3a84c65 service placement] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:45:29.949967 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1415/2875] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:45:29 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:45:33.270852 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c775bfd5-6301-4206-aa9d-acd601fbdc17 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:45:33.281719 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c775bfd5-6301-4206-aa9d-acd601fbdc17 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:45:33.285483 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b64306-dc36-49ba-86da-351b079a3cdb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eaf19a259c2741cea375a38763e33cdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:45:33.288334 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b64306-dc36-49ba-86da-351b079a3cdb service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:45:33.288334 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b64306-dc36-49ba-86da-351b079a3cdb service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:45:33.288334 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b64306-dc36-49ba-86da-351b079a3cdb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:45:33.288815 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-e9b64306-dc36-49ba-86da-351b079a3cdb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:45:33.298530 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b64306-dc36-49ba-86da-351b079a3cdb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:45:33.299989 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b64306-dc36-49ba-86da-351b079a3cdb service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:45:33.308524 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1461/2876] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:45:33 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:45:34.026183 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d894a863-ae92-436b-b74a-c682d3a84c65 service placement] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:45:34.036557 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d894a863-ae92-436b-b74a-c682d3a84c65 service placement] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:45:34.040211 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c1da49-77d2-433b-8db7-e67eb5fd75de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eaf19a259c2741cea375a38763e33cdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:45:34.040854 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c1da49-77d2-433b-8db7-e67eb5fd75de service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:45:34.045592 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c1da49-77d2-433b-8db7-e67eb5fd75de service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:45:34.045592 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c1da49-77d2-433b-8db7-e67eb5fd75de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:45:34.051888 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c1da49-77d2-433b-8db7-e67eb5fd75de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:45:34.053300 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c1da49-77d2-433b-8db7-e67eb5fd75de service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:45:34.064097 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1416/2877] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:45:34 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:45:43.304909 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=31 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:45:44.059503 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=38 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:45:57.952085 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b64306-dc36-49ba-86da-351b079a3cdb service neutron] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:45:57.957356 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-e9b64306-dc36-49ba-86da-351b079a3cdb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:45:57.973999 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b64306-dc36-49ba-86da-351b079a3cdb service neutron] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:45:57.982049 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2ae93d-f64e-4c8b-851a-c442ac1dd495 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80b138831bc54c7b986d50092d62d4af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:45:57.982837 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2ae93d-f64e-4c8b-851a-c442ac1dd495 service nova] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:45:57.983190 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2ae93d-f64e-4c8b-851a-c442ac1dd495 service nova] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:45:57.983493 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2ae93d-f64e-4c8b-851a-c442ac1dd495 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:45:57.985490 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-3f2ae93d-f64e-4c8b-851a-c442ac1dd495 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:45:57.999977 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2ae93d-f64e-4c8b-851a-c442ac1dd495 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:45:58.001598 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2ae93d-f64e-4c8b-851a-c442ac1dd495 service nova] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:45:58.020018 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1462/2878] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:45:57 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:46:08.010841 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=6 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:46:18.851291 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c1da49-77d2-433b-8db7-e67eb5fd75de service neutron] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:46:18.860285 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c1da49-77d2-433b-8db7-e67eb5fd75de service neutron] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:46:18.863946 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52574bc8-b851-4a97-a462-395baa993496 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eaf19a259c2741cea375a38763e33cdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:46:18.864581 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52574bc8-b851-4a97-a462-395baa993496 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:46:18.864864 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52574bc8-b851-4a97-a462-395baa993496 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:46:18.865141 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52574bc8-b851-4a97-a462-395baa993496 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:46:18.867447 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-52574bc8-b851-4a97-a462-395baa993496 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:46:18.931810 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52574bc8-b851-4a97-a462-395baa993496 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:46:18.933252 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52574bc8-b851-4a97-a462-395baa993496 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:46:18.943202 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1417/2879] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:46:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:46:22.915505 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2ae93d-f64e-4c8b-851a-c442ac1dd495 service nova] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:46:22.930837 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2ae93d-f64e-4c8b-851a-c442ac1dd495 service nova] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:46:22.936724 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b595858c-5fe8-42b8-8148-8b2ce83d17d6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88936b31709b429f8b6777c673ed7b8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:46:22.937481 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b595858c-5fe8-42b8-8148-8b2ce83d17d6 service placement] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:46:22.937643 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b595858c-5fe8-42b8-8148-8b2ce83d17d6 service placement] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:46:22.938395 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b595858c-5fe8-42b8-8148-8b2ce83d17d6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:46:22.940266 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-b595858c-5fe8-42b8-8148-8b2ce83d17d6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:46:22.953789 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b595858c-5fe8-42b8-8148-8b2ce83d17d6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:46:22.955447 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b595858c-5fe8-42b8-8148-8b2ce83d17d6 service placement] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:46:22.966471 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1463/2880] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:46:22 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:46:26.948120 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52574bc8-b851-4a97-a462-395baa993496 service neutron] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:46:26.949666 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-52574bc8-b851-4a97-a462-395baa993496 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:46:26.965930 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52574bc8-b851-4a97-a462-395baa993496 service neutron] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:46:26.970916 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f212eb47-92c6-482a-a1d6-125c34df9cae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:46:26.971802 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f212eb47-92c6-482a-a1d6-125c34df9cae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:46:26.972131 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f212eb47-92c6-482a-a1d6-125c34df9cae admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:46:26.972415 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f212eb47-92c6-482a-a1d6-125c34df9cae admin admin] PATH_INFO: `/v3/users/41a452eca3af46b09f56a80fa8652247` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:46:26.973606 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f212eb47-92c6-482a-a1d6-125c34df9cae admin admin] RBAC: Authorizing `identity:delete_user(user_id=41a452eca3af46b09f56a80fa8652247)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:46:26.976011 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f212eb47-92c6-482a-a1d6-125c34df9cae admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:46:27.238734 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-f212eb47-92c6-482a-a1d6-125c34df9cae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41a452eca3af46b09f56a80fa8652247'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:46:27.305986 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-f212eb47-92c6-482a-a1d6-125c34df9cae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41a452eca3af46b09f56a80fa8652247'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:46:27.306657 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-f212eb47-92c6-482a-a1d6-125c34df9cae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41a452eca3af46b09f56a80fa8652247'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:46:27.320278 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-f212eb47-92c6-482a-a1d6-125c34df9cae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41a452eca3af46b09f56a80fa8652247'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:46:27.320934 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-f212eb47-92c6-482a-a1d6-125c34df9cae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41a452eca3af46b09f56a80fa8652247'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:46:27.333323 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1418/2881] 10.222.0.66 () {58 vars in 1223 bytes} [Mon Jul 18 02:46:26 2022] DELETE /identity/v3/users/41a452eca3af46b09f56a80fa8652247 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:46:27.622931 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b595858c-5fe8-42b8-8148-8b2ce83d17d6 service placement] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:46:27.634381 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b595858c-5fe8-42b8-8148-8b2ce83d17d6 service placement] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:46:27.699948 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bad33a-4fa5-46da-9d05-f22ad011b555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:46:27.700813 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bad33a-4fa5-46da-9d05-f22ad011b555 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:46:27.701093 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bad33a-4fa5-46da-9d05-f22ad011b555 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:46:27.701365 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bad33a-4fa5-46da-9d05-f22ad011b555 admin admin] PATH_INFO: `/v3/projects/807f88779cf842f3bbe09615d470f44c` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:46:27.702557 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bad33a-4fa5-46da-9d05-f22ad011b555 admin admin] RBAC: Authorizing `identity:delete_project(project_id=807f88779cf842f3bbe09615d470f44c)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:46:27.704441 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bad33a-4fa5-46da-9d05-f22ad011b555 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:46:27.860749 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-f4bad33a-4fa5-46da-9d05-f22ad011b555 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '807f88779cf842f3bbe09615d470f44c', 'request_id': 'req-f4bad33a-4fa5-46da-9d05-f22ad011b555'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:46:27.874077 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-f4bad33a-4fa5-46da-9d05-f22ad011b555 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '807f88779cf842f3bbe09615d470f44c', 'request_id': 'req-f4bad33a-4fa5-46da-9d05-f22ad011b555'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:46:28.002730 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-f4bad33a-4fa5-46da-9d05-f22ad011b555 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '807f88779cf842f3bbe09615d470f44c', 'request_id': 'req-f4bad33a-4fa5-46da-9d05-f22ad011b555'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:46:28.014917 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-f4bad33a-4fa5-46da-9d05-f22ad011b555 admin admin] The token cache is being invalidate because project 807f88779cf842f3bbe09615d470f44c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 02:46:28.015204 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-f4bad33a-4fa5-46da-9d05-f22ad011b555 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:46:28.015833 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-f4bad33a-4fa5-46da-9d05-f22ad011b555 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:46:28.017393 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1464/2882] 10.222.0.66 () {58 vars in 1232 bytes} [Mon Jul 18 02:46:27 2022] DELETE /identity/v3/projects/807f88779cf842f3bbe09615d470f44c => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:46:28.021359 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f212eb47-92c6-482a-a1d6-125c34df9cae admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:46:28.023381 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-f212eb47-92c6-482a-a1d6-125c34df9cae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:46:28.036523 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f212eb47-92c6-482a-a1d6-125c34df9cae admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:46:28.092841 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14a831e-dd8e-40b4-acd6-4777aae7294b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:46:28.093712 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14a831e-dd8e-40b4-acd6-4777aae7294b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:46:28.093996 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14a831e-dd8e-40b4-acd6-4777aae7294b admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:46:28.094306 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14a831e-dd8e-40b4-acd6-4777aae7294b admin admin] PATH_INFO: `/v3/users/ce8b774659014eccb994e023afca7b9f` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:46:28.095480 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14a831e-dd8e-40b4-acd6-4777aae7294b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce8b774659014eccb994e023afca7b9f)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:46:28.097408 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14a831e-dd8e-40b4-acd6-4777aae7294b admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:46:28.103420 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:46:28.336879 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-d14a831e-dd8e-40b4-acd6-4777aae7294b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce8b774659014eccb994e023afca7b9f'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:46:28.427034 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-d14a831e-dd8e-40b4-acd6-4777aae7294b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce8b774659014eccb994e023afca7b9f'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:46:28.427659 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-d14a831e-dd8e-40b4-acd6-4777aae7294b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce8b774659014eccb994e023afca7b9f'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:46:28.440638 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-d14a831e-dd8e-40b4-acd6-4777aae7294b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce8b774659014eccb994e023afca7b9f'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:46:28.441592 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-d14a831e-dd8e-40b4-acd6-4777aae7294b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce8b774659014eccb994e023afca7b9f'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:46:28.454614 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1419/2883] 10.222.0.66 () {58 vars in 1223 bytes} [Mon Jul 18 02:46:28 2022] DELETE /identity/v3/users/ce8b774659014eccb994e023afca7b9f => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:46:28.747717 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bad33a-4fa5-46da-9d05-f22ad011b555 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:46:28.749211 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-f4bad33a-4fa5-46da-9d05-f22ad011b555 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:46:28.765088 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bad33a-4fa5-46da-9d05-f22ad011b555 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:46:28.823710 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7bae9f-de5b-4d73-9915-c2ce4fb478aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:46:28.824624 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7bae9f-de5b-4d73-9915-c2ce4fb478aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:46:28.824865 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7bae9f-de5b-4d73-9915-c2ce4fb478aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:46:28.825136 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7bae9f-de5b-4d73-9915-c2ce4fb478aa admin admin] PATH_INFO: `/v3/projects/d447d08d2f9c48caaed6d78c2dc46e2b` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:46:28.829197 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7bae9f-de5b-4d73-9915-c2ce4fb478aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=d447d08d2f9c48caaed6d78c2dc46e2b)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:46:28.829197 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7bae9f-de5b-4d73-9915-c2ce4fb478aa admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:46:28.850300 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:46:28.988102 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-dd7bae9f-de5b-4d73-9915-c2ce4fb478aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd447d08d2f9c48caaed6d78c2dc46e2b', 'request_id': 'req-dd7bae9f-de5b-4d73-9915-c2ce4fb478aa'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:46:28.994113 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-dd7bae9f-de5b-4d73-9915-c2ce4fb478aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd447d08d2f9c48caaed6d78c2dc46e2b', 'request_id': 'req-dd7bae9f-de5b-4d73-9915-c2ce4fb478aa'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:46:29.089608 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-dd7bae9f-de5b-4d73-9915-c2ce4fb478aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd447d08d2f9c48caaed6d78c2dc46e2b', 'request_id': 'req-dd7bae9f-de5b-4d73-9915-c2ce4fb478aa'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:46:29.102520 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-dd7bae9f-de5b-4d73-9915-c2ce4fb478aa admin admin] The token cache is being invalidate because project d447d08d2f9c48caaed6d78c2dc46e2b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 02:46:29.104454 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-dd7bae9f-de5b-4d73-9915-c2ce4fb478aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:46:29.104454 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-dd7bae9f-de5b-4d73-9915-c2ce4fb478aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:46:29.105142 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1465/2884] 10.222.0.66 () {58 vars in 1232 bytes} [Mon Jul 18 02:46:28 2022] DELETE /identity/v3/projects/d447d08d2f9c48caaed6d78c2dc46e2b => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:46:32.643715 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14a831e-dd8e-40b4-acd6-4777aae7294b admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:46:32.645233 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-d14a831e-dd8e-40b4-acd6-4777aae7294b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:46:32.721632 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14a831e-dd8e-40b4-acd6-4777aae7294b admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:46:32.786379 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52e9e23-409d-420b-972f-2645bd1f8888 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80b138831bc54c7b986d50092d62d4af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:46:32.787004 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52e9e23-409d-420b-972f-2645bd1f8888 service nova] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:46:32.787817 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52e9e23-409d-420b-972f-2645bd1f8888 service nova] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:46:32.788101 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52e9e23-409d-420b-972f-2645bd1f8888 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:46:32.789924 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-f52e9e23-409d-420b-972f-2645bd1f8888 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:46:32.802601 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52e9e23-409d-420b-972f-2645bd1f8888 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:46:32.804910 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52e9e23-409d-420b-972f-2645bd1f8888 service nova] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:46:32.816023 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1420/2885] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:46:32 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:46:33.268102 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7bae9f-de5b-4d73-9915-c2ce4fb478aa admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:46:33.269659 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-dd7bae9f-de5b-4d73-9915-c2ce4fb478aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:46:33.331544 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7bae9f-de5b-4d73-9915-c2ce4fb478aa admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:46:33.335825 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40603e5c-41ac-41e6-9d5e-fd569d7b9b47 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eaf19a259c2741cea375a38763e33cdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:46:33.336469 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40603e5c-41ac-41e6-9d5e-fd569d7b9b47 service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:46:33.336762 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40603e5c-41ac-41e6-9d5e-fd569d7b9b47 service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:46:33.337036 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40603e5c-41ac-41e6-9d5e-fd569d7b9b47 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:46:33.338809 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-40603e5c-41ac-41e6-9d5e-fd569d7b9b47 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:46:33.348419 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40603e5c-41ac-41e6-9d5e-fd569d7b9b47 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:46:33.350673 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40603e5c-41ac-41e6-9d5e-fd569d7b9b47 service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:46:33.358712 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1466/2886] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:46:33 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:46:33.434195 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52e9e23-409d-420b-972f-2645bd1f8888 service nova] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:46:33.444023 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52e9e23-409d-420b-972f-2645bd1f8888 service nova] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:46:33.447646 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe158e8d-2de8-41ec-b4d7-921297b86a74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eaf19a259c2741cea375a38763e33cdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:46:33.448339 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe158e8d-2de8-41ec-b4d7-921297b86a74 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:46:33.448631 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe158e8d-2de8-41ec-b4d7-921297b86a74 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:46:33.448948 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe158e8d-2de8-41ec-b4d7-921297b86a74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:46:33.450738 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-fe158e8d-2de8-41ec-b4d7-921297b86a74 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:46:33.462715 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:46:33.465041 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe158e8d-2de8-41ec-b4d7-921297b86a74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:46:33.466530 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe158e8d-2de8-41ec-b4d7-921297b86a74 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:46:33.478266 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1421/2887] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:46:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:46:42.958316 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40603e5c-41ac-41e6-9d5e-fd569d7b9b47 service neutron] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:46:42.962274 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-40603e5c-41ac-41e6-9d5e-fd569d7b9b47 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:46:43.026982 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40603e5c-41ac-41e6-9d5e-fd569d7b9b47 service neutron] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:46:43.031861 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe5deaa-a331-4d88-b1ff-83e09d5ece7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:46:43.032762 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe5deaa-a331-4d88-b1ff-83e09d5ece7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:46:43.033045 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe5deaa-a331-4d88-b1ff-83e09d5ece7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:46:43.033323 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe5deaa-a331-4d88-b1ff-83e09d5ece7d admin admin] PATH_INFO: `/v3/users/b065087a79524114b5cacb8c5fd3254f` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:46:43.034589 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe5deaa-a331-4d88-b1ff-83e09d5ece7d admin admin] RBAC: Authorizing `identity:delete_user(user_id=b065087a79524114b5cacb8c5fd3254f)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:46:43.036596 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe5deaa-a331-4d88-b1ff-83e09d5ece7d admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:46:43.194961 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-dfe5deaa-a331-4d88-b1ff-83e09d5ece7d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b065087a79524114b5cacb8c5fd3254f'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:46:43.198753 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:46:43.259714 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-dfe5deaa-a331-4d88-b1ff-83e09d5ece7d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b065087a79524114b5cacb8c5fd3254f'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:46:43.267367 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-dfe5deaa-a331-4d88-b1ff-83e09d5ece7d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b065087a79524114b5cacb8c5fd3254f'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:46:43.271919 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-dfe5deaa-a331-4d88-b1ff-83e09d5ece7d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b065087a79524114b5cacb8c5fd3254f'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:46:43.279934 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-dfe5deaa-a331-4d88-b1ff-83e09d5ece7d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b065087a79524114b5cacb8c5fd3254f'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:46:43.285880 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1467/2888] 10.222.0.66 () {58 vars in 1223 bytes} [Mon Jul 18 02:46:42 2022] DELETE /identity/v3/users/b065087a79524114b5cacb8c5fd3254f => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:46:43.558821 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe158e8d-2de8-41ec-b4d7-921297b86a74 service neutron] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:46:43.569832 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe158e8d-2de8-41ec-b4d7-921297b86a74 service neutron] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:46:43.630510 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8559cc9b-4b86-43c3-82e5-489225a956bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:46:43.631380 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8559cc9b-4b86-43c3-82e5-489225a956bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:46:43.631661 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8559cc9b-4b86-43c3-82e5-489225a956bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:46:43.631941 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8559cc9b-4b86-43c3-82e5-489225a956bf admin admin] PATH_INFO: `/v3/projects/ad2b132c879f4f5e90593713a3f472c1` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:46:43.633103 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8559cc9b-4b86-43c3-82e5-489225a956bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad2b132c879f4f5e90593713a3f472c1)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:46:43.635046 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8559cc9b-4b86-43c3-82e5-489225a956bf admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:46:43.787411 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-8559cc9b-4b86-43c3-82e5-489225a956bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad2b132c879f4f5e90593713a3f472c1', 'request_id': 'req-8559cc9b-4b86-43c3-82e5-489225a956bf'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:46:43.794190 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-8559cc9b-4b86-43c3-82e5-489225a956bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad2b132c879f4f5e90593713a3f472c1', 'request_id': 'req-8559cc9b-4b86-43c3-82e5-489225a956bf'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:46:43.800904 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-8559cc9b-4b86-43c3-82e5-489225a956bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad2b132c879f4f5e90593713a3f472c1', 'request_id': 'req-8559cc9b-4b86-43c3-82e5-489225a956bf'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:46:43.882272 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-8559cc9b-4b86-43c3-82e5-489225a956bf admin admin] The token cache is being invalidate because project ad2b132c879f4f5e90593713a3f472c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 02:46:43.882562 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-8559cc9b-4b86-43c3-82e5-489225a956bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:46:43.883849 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-8559cc9b-4b86-43c3-82e5-489225a956bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:46:43.885620 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1422/2889] 10.222.0.66 () {58 vars in 1232 bytes} [Mon Jul 18 02:46:43 2022] DELETE /identity/v3/projects/ad2b132c879f4f5e90593713a3f472c1 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:46:51.088148 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe5deaa-a331-4d88-b1ff-83e09d5ece7d admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:46:51.089916 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-dfe5deaa-a331-4d88-b1ff-83e09d5ece7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:46:51.193333 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe5deaa-a331-4d88-b1ff-83e09d5ece7d admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:46:51.272950 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d80095-6da9-4e76-92ca-6930dcd85ca0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80b138831bc54c7b986d50092d62d4af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:46:51.273805 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d80095-6da9-4e76-92ca-6930dcd85ca0 service nova] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:46:51.274046 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d80095-6da9-4e76-92ca-6930dcd85ca0 service nova] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:46:51.274327 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d80095-6da9-4e76-92ca-6930dcd85ca0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:46:51.276394 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-d1d80095-6da9-4e76-92ca-6930dcd85ca0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:46:51.289062 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d80095-6da9-4e76-92ca-6930dcd85ca0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:46:51.290747 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d80095-6da9-4e76-92ca-6930dcd85ca0 service nova] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:46:51.311001 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1468/2890] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:46:51 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:46:53.879034 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=53,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:47:00.054029 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c94e79c-d50d-4e27-aa5c-125d331cff9f None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:47:00.057351 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c94e79c-d50d-4e27-aa5c-125d331cff9f None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:47:00.057351 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c94e79c-d50d-4e27-aa5c-125d331cff9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:47:00.103685 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-0c94e79c-d50d-4e27-aa5c-125d331cff9f None None] MFA Rules not processed for user `768f872096584d4093fc616d547aabb6`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 02:47:00.104822 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-0c94e79c-d50d-4e27-aa5c-125d331cff9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:47:00.166963 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1423/2891] 10.222.0.66 () {58 vars in 967 bytes} [Mon Jul 18 02:47:00 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 02:47:00.207332 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d80095-6da9-4e76-92ca-6930dcd85ca0 service nova] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:47:00.208717 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-d1d80095-6da9-4e76-92ca-6930dcd85ca0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:47:00.264152 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:47:00.296481 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d80095-6da9-4e76-92ca-6930dcd85ca0 service nova] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:47:00.301409 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d0f40e-c65f-43bb-8078-1f52773e9256 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4300622cbd347dd8f5227bb960e98d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:47:00.302043 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d0f40e-c65f-43bb-8078-1f52773e9256 service swift] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:47:00.302360 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d0f40e-c65f-43bb-8078-1f52773e9256 service swift] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:47:00.302636 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d0f40e-c65f-43bb-8078-1f52773e9256 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:47:00.310634 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d0f40e-c65f-43bb-8078-1f52773e9256 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:47:00.312205 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d0f40e-c65f-43bb-8078-1f52773e9256 service swift] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:47:00.320318 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1469/2892] 10.222.0.66 () {60 vars in 1384 bytes} [Mon Jul 18 02:47:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 02:47:08.826394 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c94e79c-d50d-4e27-aa5c-125d331cff9f None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:47:08.827905 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-0c94e79c-d50d-4e27-aa5c-125d331cff9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:47:08.895916 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c94e79c-d50d-4e27-aa5c-125d331cff9f None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:47:08.900615 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8fcf50-8900-4885-9f7d-aca0f939d65f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:47:08.901479 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8fcf50-8900-4885-9f7d-aca0f939d65f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:47:08.901765 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8fcf50-8900-4885-9f7d-aca0f939d65f admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:47:08.902045 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8fcf50-8900-4885-9f7d-aca0f939d65f admin admin] PATH_INFO: `/v3/users/3277db965cea4f23b463836d26990e1c` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:47:08.903304 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8fcf50-8900-4885-9f7d-aca0f939d65f admin admin] RBAC: Authorizing `identity:delete_user(user_id=3277db965cea4f23b463836d26990e1c)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:47:08.905238 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8fcf50-8900-4885-9f7d-aca0f939d65f admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:47:09.124088 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-ec8fcf50-8900-4885-9f7d-aca0f939d65f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3277db965cea4f23b463836d26990e1c'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:47:09.191026 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-ec8fcf50-8900-4885-9f7d-aca0f939d65f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3277db965cea4f23b463836d26990e1c'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:47:09.191670 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-ec8fcf50-8900-4885-9f7d-aca0f939d65f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3277db965cea4f23b463836d26990e1c'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:47:09.203938 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-ec8fcf50-8900-4885-9f7d-aca0f939d65f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3277db965cea4f23b463836d26990e1c'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:47:09.205988 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-ec8fcf50-8900-4885-9f7d-aca0f939d65f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3277db965cea4f23b463836d26990e1c'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:47:09.206682 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:47:09.217826 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1424/2893] 10.222.0.66 () {58 vars in 1223 bytes} [Mon Jul 18 02:47:08 2022] DELETE /identity/v3/users/3277db965cea4f23b463836d26990e1c => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:47:09.443790 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d0f40e-c65f-43bb-8078-1f52773e9256 service swift] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:47:09.455348 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d0f40e-c65f-43bb-8078-1f52773e9256 service swift] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:47:09.514100 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad444de2-9bc6-4446-bfcf-997944ae5b3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:47:09.515008 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad444de2-9bc6-4446-bfcf-997944ae5b3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:47:09.515292 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad444de2-9bc6-4446-bfcf-997944ae5b3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:47:09.515570 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad444de2-9bc6-4446-bfcf-997944ae5b3d admin admin] PATH_INFO: `/v3/projects/73679a05b6dc4583842dacfeb898876f` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:47:09.516766 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad444de2-9bc6-4446-bfcf-997944ae5b3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=73679a05b6dc4583842dacfeb898876f)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:47:09.518718 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad444de2-9bc6-4446-bfcf-997944ae5b3d admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:47:09.643048 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-ad444de2-9bc6-4446-bfcf-997944ae5b3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73679a05b6dc4583842dacfeb898876f', 'request_id': 'req-ad444de2-9bc6-4446-bfcf-997944ae5b3d'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:47:09.648614 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-ad444de2-9bc6-4446-bfcf-997944ae5b3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73679a05b6dc4583842dacfeb898876f', 'request_id': 'req-ad444de2-9bc6-4446-bfcf-997944ae5b3d'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:47:09.733503 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-ad444de2-9bc6-4446-bfcf-997944ae5b3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73679a05b6dc4583842dacfeb898876f', 'request_id': 'req-ad444de2-9bc6-4446-bfcf-997944ae5b3d'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:47:09.744708 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-ad444de2-9bc6-4446-bfcf-997944ae5b3d admin admin] The token cache is being invalidate because project 73679a05b6dc4583842dacfeb898876f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 02:47:09.744994 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-ad444de2-9bc6-4446-bfcf-997944ae5b3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:47:09.745617 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-ad444de2-9bc6-4446-bfcf-997944ae5b3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:47:09.747696 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1470/2894] 10.222.0.66 () {58 vars in 1232 bytes} [Mon Jul 18 02:47:09 2022] DELETE /identity/v3/projects/73679a05b6dc4583842dacfeb898876f => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:47:09.751799 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8fcf50-8900-4885-9f7d-aca0f939d65f admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:47:09.753173 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-ec8fcf50-8900-4885-9f7d-aca0f939d65f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:47:09.766977 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8fcf50-8900-4885-9f7d-aca0f939d65f admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:47:09.823583 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d1d1a4-42af-4a51-b759-11d914871033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:47:09.825379 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d1d1a4-42af-4a51-b759-11d914871033 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:47:09.825666 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d1d1a4-42af-4a51-b759-11d914871033 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:47:09.825944 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d1d1a4-42af-4a51-b759-11d914871033 admin admin] PATH_INFO: `/v3/users/764dad0085f846a0bc6855b94bf782d5` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:47:09.827174 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d1d1a4-42af-4a51-b759-11d914871033 admin admin] RBAC: Authorizing `identity:delete_user(user_id=764dad0085f846a0bc6855b94bf782d5)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:47:09.829108 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d1d1a4-42af-4a51-b759-11d914871033 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:47:10.045069 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-d9d1d1a4-42af-4a51-b759-11d914871033 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '764dad0085f846a0bc6855b94bf782d5'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:47:10.122993 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-d9d1d1a4-42af-4a51-b759-11d914871033 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '764dad0085f846a0bc6855b94bf782d5'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:47:10.123606 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-d9d1d1a4-42af-4a51-b759-11d914871033 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '764dad0085f846a0bc6855b94bf782d5'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:47:10.135892 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-d9d1d1a4-42af-4a51-b759-11d914871033 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '764dad0085f846a0bc6855b94bf782d5'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:47:10.138012 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-d9d1d1a4-42af-4a51-b759-11d914871033 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '764dad0085f846a0bc6855b94bf782d5'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:47:10.148421 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1425/2895] 10.222.0.66 () {58 vars in 1223 bytes} [Mon Jul 18 02:47:09 2022] DELETE /identity/v3/users/764dad0085f846a0bc6855b94bf782d5 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:47:10.380734 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad444de2-9bc6-4446-bfcf-997944ae5b3d admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:47:10.382254 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-ad444de2-9bc6-4446-bfcf-997944ae5b3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:47:10.395028 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad444de2-9bc6-4446-bfcf-997944ae5b3d admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:47:10.444011 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:47:10.453224 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3fb396-03d3-4c2e-ac54-c42793459691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:47:10.454105 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3fb396-03d3-4c2e-ac54-c42793459691 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:47:10.454440 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3fb396-03d3-4c2e-ac54-c42793459691 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:47:10.455440 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3fb396-03d3-4c2e-ac54-c42793459691 admin admin] PATH_INFO: `/v3/projects/59d0c1f5793f4ac3aa9567bd99cf987d` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:47:10.456047 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3fb396-03d3-4c2e-ac54-c42793459691 admin admin] RBAC: Authorizing `identity:delete_project(project_id=59d0c1f5793f4ac3aa9567bd99cf987d)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:47:10.458158 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3fb396-03d3-4c2e-ac54-c42793459691 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:47:10.562224 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-ad3fb396-03d3-4c2e-ac54-c42793459691 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59d0c1f5793f4ac3aa9567bd99cf987d', 'request_id': 'req-ad3fb396-03d3-4c2e-ac54-c42793459691'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:47:10.568132 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-ad3fb396-03d3-4c2e-ac54-c42793459691 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59d0c1f5793f4ac3aa9567bd99cf987d', 'request_id': 'req-ad3fb396-03d3-4c2e-ac54-c42793459691'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:47:10.642461 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-ad3fb396-03d3-4c2e-ac54-c42793459691 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59d0c1f5793f4ac3aa9567bd99cf987d', 'request_id': 'req-ad3fb396-03d3-4c2e-ac54-c42793459691'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:47:10.654695 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-ad3fb396-03d3-4c2e-ac54-c42793459691 admin admin] The token cache is being invalidate because project 59d0c1f5793f4ac3aa9567bd99cf987d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 02:47:10.654989 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-ad3fb396-03d3-4c2e-ac54-c42793459691 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:47:10.655625 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-ad3fb396-03d3-4c2e-ac54-c42793459691 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:47:10.657171 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1471/2896] 10.222.0.66 () {58 vars in 1232 bytes} [Mon Jul 18 02:47:10 2022] DELETE /identity/v3/projects/59d0c1f5793f4ac3aa9567bd99cf987d => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:47:20.145125 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=57,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:47:20.651138 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=27,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:48:05.132382 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d1d1a4-42af-4a51-b759-11d914871033 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:48:05.134609 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-d9d1d1a4-42af-4a51-b759-11d914871033 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:48:05.192052 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d1d1a4-42af-4a51-b759-11d914871033 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:48:05.240867 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57bba10-8a9e-42ca-86a7-a59b04b59d5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eaf19a259c2741cea375a38763e33cdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:48:05.241522 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57bba10-8a9e-42ca-86a7-a59b04b59d5f service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:48:05.241807 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57bba10-8a9e-42ca-86a7-a59b04b59d5f service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:48:05.242099 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57bba10-8a9e-42ca-86a7-a59b04b59d5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:48:05.244572 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-e57bba10-8a9e-42ca-86a7-a59b04b59d5f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:48:05.301895 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57bba10-8a9e-42ca-86a7-a59b04b59d5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:48:05.303368 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57bba10-8a9e-42ca-86a7-a59b04b59d5f service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:48:05.312894 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1426/2897] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:48:05 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:48:11.399523 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3fb396-03d3-4c2e-ac54-c42793459691 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:48:11.401660 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-ad3fb396-03d3-4c2e-ac54-c42793459691 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:48:11.413878 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3fb396-03d3-4c2e-ac54-c42793459691 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:48:11.417494 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1698a1-e41e-4252-92eb-af43fbd76a83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eaf19a259c2741cea375a38763e33cdc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:48:11.418169 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1698a1-e41e-4252-92eb-af43fbd76a83 service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:48:11.418456 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1698a1-e41e-4252-92eb-af43fbd76a83 service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:48:11.418737 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1698a1-e41e-4252-92eb-af43fbd76a83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:48:11.420462 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-2c1698a1-e41e-4252-92eb-af43fbd76a83 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:48:11.485883 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1698a1-e41e-4252-92eb-af43fbd76a83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:48:11.487389 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1698a1-e41e-4252-92eb-af43fbd76a83 service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:48:11.497051 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1472/2898] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:48:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:48:15.109208 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57bba10-8a9e-42ca-86a7-a59b04b59d5f service neutron] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:48:15.110714 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-e57bba10-8a9e-42ca-86a7-a59b04b59d5f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:48:15.186099 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57bba10-8a9e-42ca-86a7-a59b04b59d5f service neutron] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:48:15.191422 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a4f731-2d58-4dc8-b302-592d87beb21e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88936b31709b429f8b6777c673ed7b8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:48:15.192089 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a4f731-2d58-4dc8-b302-592d87beb21e service placement] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:48:15.192373 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a4f731-2d58-4dc8-b302-592d87beb21e service placement] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:48:15.192646 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a4f731-2d58-4dc8-b302-592d87beb21e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:48:15.194447 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-46a4f731-2d58-4dc8-b302-592d87beb21e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:48:15.206451 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a4f731-2d58-4dc8-b302-592d87beb21e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:48:15.208142 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a4f731-2d58-4dc8-b302-592d87beb21e service placement] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:48:15.214435 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:48:15.218997 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1427/2899] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:48:15 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:48:18.927292 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1698a1-e41e-4252-92eb-af43fbd76a83 service neutron] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:48:18.928799 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-2c1698a1-e41e-4252-92eb-af43fbd76a83 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:48:18.943112 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1698a1-e41e-4252-92eb-af43fbd76a83 service neutron] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:48:18.947779 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df2fe35-af63-4079-9172-2100f929d0d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:48:18.948708 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df2fe35-af63-4079-9172-2100f929d0d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:48:18.948997 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df2fe35-af63-4079-9172-2100f929d0d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:48:18.949276 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df2fe35-af63-4079-9172-2100f929d0d1 admin admin] PATH_INFO: `/v3/users/681d0f989d7d459e83a32066fd5ed0a2` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:48:18.950490 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df2fe35-af63-4079-9172-2100f929d0d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=681d0f989d7d459e83a32066fd5ed0a2)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:48:18.953271 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df2fe35-af63-4079-9172-2100f929d0d1 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:48:19.214113 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-6df2fe35-af63-4079-9172-2100f929d0d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '681d0f989d7d459e83a32066fd5ed0a2'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:48:19.331341 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-6df2fe35-af63-4079-9172-2100f929d0d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '681d0f989d7d459e83a32066fd5ed0a2'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:48:19.339335 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-6df2fe35-af63-4079-9172-2100f929d0d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '681d0f989d7d459e83a32066fd5ed0a2'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:48:19.339973 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-6df2fe35-af63-4079-9172-2100f929d0d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '681d0f989d7d459e83a32066fd5ed0a2'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:48:19.351695 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-6df2fe35-af63-4079-9172-2100f929d0d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '681d0f989d7d459e83a32066fd5ed0a2'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:48:19.358436 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1473/2900] 10.222.0.66 () {58 vars in 1223 bytes} [Mon Jul 18 02:48:18 2022] DELETE /identity/v3/users/681d0f989d7d459e83a32066fd5ed0a2 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:48:19.652669 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a4f731-2d58-4dc8-b302-592d87beb21e service placement] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:48:19.663873 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a4f731-2d58-4dc8-b302-592d87beb21e service placement] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:48:19.723471 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc90b331-20cf-4ed7-a86f-108ed04efff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:48:19.724349 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc90b331-20cf-4ed7-a86f-108ed04efff1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:48:19.724612 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc90b331-20cf-4ed7-a86f-108ed04efff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:48:19.724900 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc90b331-20cf-4ed7-a86f-108ed04efff1 admin admin] PATH_INFO: `/v3/projects/2fe363fe43ef4fd9aec026c4cdc3e18e` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:48:19.726101 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc90b331-20cf-4ed7-a86f-108ed04efff1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2fe363fe43ef4fd9aec026c4cdc3e18e)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:48:19.728073 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc90b331-20cf-4ed7-a86f-108ed04efff1 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:48:19.839863 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-cc90b331-20cf-4ed7-a86f-108ed04efff1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2fe363fe43ef4fd9aec026c4cdc3e18e', 'request_id': 'req-cc90b331-20cf-4ed7-a86f-108ed04efff1'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:48:19.845170 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-cc90b331-20cf-4ed7-a86f-108ed04efff1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2fe363fe43ef4fd9aec026c4cdc3e18e', 'request_id': 'req-cc90b331-20cf-4ed7-a86f-108ed04efff1'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:48:19.851144 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-cc90b331-20cf-4ed7-a86f-108ed04efff1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2fe363fe43ef4fd9aec026c4cdc3e18e', 'request_id': 'req-cc90b331-20cf-4ed7-a86f-108ed04efff1'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:48:19.925204 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-cc90b331-20cf-4ed7-a86f-108ed04efff1 admin admin] The token cache is being invalidate because project 2fe363fe43ef4fd9aec026c4cdc3e18e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 02:48:19.925496 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-cc90b331-20cf-4ed7-a86f-108ed04efff1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:48:19.926538 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-cc90b331-20cf-4ed7-a86f-108ed04efff1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:48:19.927777 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1428/2901] 10.222.0.66 () {58 vars in 1232 bytes} [Mon Jul 18 02:48:19 2022] DELETE /identity/v3/projects/2fe363fe43ef4fd9aec026c4cdc3e18e => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:48:19.932392 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df2fe35-af63-4079-9172-2100f929d0d1 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:48:19.933973 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-6df2fe35-af63-4079-9172-2100f929d0d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:48:19.950831 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df2fe35-af63-4079-9172-2100f929d0d1 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:48:20.014182 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda6ef5d-97ad-4381-aeff-9ab49e4d3ef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:48:20.015216 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda6ef5d-97ad-4381-aeff-9ab49e4d3ef6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:48:20.015512 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda6ef5d-97ad-4381-aeff-9ab49e4d3ef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:48:20.015795 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda6ef5d-97ad-4381-aeff-9ab49e4d3ef6 admin admin] PATH_INFO: `/v3/users/cb4702f0216b47b8ba83872eeee40173` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:48:20.017001 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda6ef5d-97ad-4381-aeff-9ab49e4d3ef6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb4702f0216b47b8ba83872eeee40173)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:48:20.018954 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda6ef5d-97ad-4381-aeff-9ab49e4d3ef6 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:48:20.024301 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:48:20.219346 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-dda6ef5d-97ad-4381-aeff-9ab49e4d3ef6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb4702f0216b47b8ba83872eeee40173'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:48:20.302343 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-dda6ef5d-97ad-4381-aeff-9ab49e4d3ef6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb4702f0216b47b8ba83872eeee40173'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:48:20.310364 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-dda6ef5d-97ad-4381-aeff-9ab49e4d3ef6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb4702f0216b47b8ba83872eeee40173'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:48:20.311558 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-dda6ef5d-97ad-4381-aeff-9ab49e4d3ef6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb4702f0216b47b8ba83872eeee40173'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:48:20.322967 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-dda6ef5d-97ad-4381-aeff-9ab49e4d3ef6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb4702f0216b47b8ba83872eeee40173'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:48:20.328747 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1474/2902] 10.222.0.66 () {58 vars in 1223 bytes} [Mon Jul 18 02:48:19 2022] DELETE /identity/v3/users/cb4702f0216b47b8ba83872eeee40173 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 02:48:20.578598 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc90b331-20cf-4ed7-a86f-108ed04efff1 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:48:20.580181 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-cc90b331-20cf-4ed7-a86f-108ed04efff1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:48:20.594448 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc90b331-20cf-4ed7-a86f-108ed04efff1 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:48:20.655732 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d628947-0113-462d-a850-db61eda156d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62dd67591e0147fe97b248079359a4de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a05e40a15656441b94f5763e4d68d305', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:48:20.656604 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d628947-0113-462d-a850-db61eda156d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:48:20.656893 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d628947-0113-462d-a850-db61eda156d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:48:20.657180 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d628947-0113-462d-a850-db61eda156d0 admin admin] PATH_INFO: `/v3/projects/8505b1f5965f4d04b1897f01c135d6ea` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:48:20.658415 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d628947-0113-462d-a850-db61eda156d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8505b1f5965f4d04b1897f01c135d6ea)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:48:20.660333 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d628947-0113-462d-a850-db61eda156d0 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:48:20.788453 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:48:20.800834 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-3d628947-0113-462d-a850-db61eda156d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8505b1f5965f4d04b1897f01c135d6ea', 'request_id': 'req-3d628947-0113-462d-a850-db61eda156d0'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:48:20.806151 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-3d628947-0113-462d-a850-db61eda156d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8505b1f5965f4d04b1897f01c135d6ea', 'request_id': 'req-3d628947-0113-462d-a850-db61eda156d0'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:48:20.812760 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-3d628947-0113-462d-a850-db61eda156d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8505b1f5965f4d04b1897f01c135d6ea', 'request_id': 'req-3d628947-0113-462d-a850-db61eda156d0'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:48:20.896322 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-3d628947-0113-462d-a850-db61eda156d0 admin admin] The token cache is being invalidate because project 8505b1f5965f4d04b1897f01c135d6ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 02:48:20.896593 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-3d628947-0113-462d-a850-db61eda156d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:48:20.897232 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-3d628947-0113-462d-a850-db61eda156d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 02:48:20.898834 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1429/2903] 10.222.0.66 () {58 vars in 1232 bytes} [Mon Jul 18 02:48:20 2022] DELETE /identity/v3/projects/8505b1f5965f4d04b1897f01c135d6ea => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 02:48:30.325251 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=23,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:48:30.966029 dv-cinder-850015-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=12,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 02:49:11.791154 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda6ef5d-97ad-4381-aeff-9ab49e4d3ef6 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 02:49:11.792646 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-dda6ef5d-97ad-4381-aeff-9ab49e4d3ef6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:49:11.862141 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda6ef5d-97ad-4381-aeff-9ab49e4d3ef6 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 02:49:11.921465 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1bdb1e-333c-4fc2-bdfe-46e471c42a9e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88936b31709b429f8b6777c673ed7b8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76f7c0ae36ae49b09dcae4c3e9a875f4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 02:49:11.922170 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1bdb1e-333c-4fc2-bdfe-46e471c42a9e service placement] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 02:49:11.922572 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1bdb1e-333c-4fc2-bdfe-46e471c42a9e service placement] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 02:49:11.922731 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1bdb1e-333c-4fc2-bdfe-46e471c42a9e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 02:49:11.924534 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-bc1bdb1e-333c-4fc2-bdfe-46e471c42a9e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 02:49:11.936095 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1bdb1e-333c-4fc2-bdfe-46e471c42a9e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 02:49:11.937676 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1bdb1e-333c-4fc2-bdfe-46e471c42a9e service placement] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 02:49:11.947961 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1475/2904] 10.222.0.66 () {60 vars in 1365 bytes} [Mon Jul 18 02:49:11 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 02:49:21.943648 dv-cinder-850015-3-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=58 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}